[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:14:41 dist-test-slave-10pc krb5kdc[16236](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:14:41 dist-test-slave-10pc krb5kdc[16236](info): set up 2 sockets
Aug 10 07:14:41 dist-test-slave-10pc krb5kdc[16236](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260810 07:14:42.873946 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:14:42 dist-test-slave-10pc krb5kdc[16236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:14:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:14:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
I20260810 07:14:43.003962 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:36347
--webserver_interface=127.15.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:36347
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:14:43.091235 16250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:14:43.091449 16250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:14:43.091471 16250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:14:43.092626 16250 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:14:43.092672 16250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:14:43.092687 16250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:14:43.092700 16250 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:14:43.092712 16250 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:14:43.094183 16250 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:36347
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:36347
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:14:43.094394 16250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:14:43.094596 16250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:14:43.099931 16256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:43.099925 16255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:43.099948 16258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:14:43.099948 16250 server_base.cc:1034] running on GCE node
I20260810 07:14:43.100335 16250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:14:43.100674 16250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:14:43.101857 16250 hybrid_clock.cc:648] HybridClock initialized: now 1786346083101879 us; error 67 us; skew 500 ppm
Aug 10 07:14:43 dist-test-slave-10pc krb5kdc[16236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346083, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:14:43.104347 16250 init.cc:377] Logged in from keytab as kudu/127.15.217.126@KRBTEST.COM (short username kudu)
I20260810 07:14:43.104880 16250 webserver.cc:460] Webserver started at http://127.15.217.126:46735/ using document root <none> and password file <none>
I20260810 07:14:43.105137 16250 fs_manager.cc:359] Metadata directory not provided
I20260810 07:14:43.105207 16250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:14:43.105373 16250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:14:43.106531 16250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "53bbb932870a4be69dbbf699f4215f8c"
format_stamp: "Formatted at 2026-08-10 07:14:43 on dist-test-slave-10pc"
server_key: "02a149ac5a3f16220e81673d74f36901"
server_key_iv: "ee403ed9a89a7279dca4dc95ade45e64"
server_key_version: "encryptionkey@0"
I20260810 07:14:43.106968 16250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "53bbb932870a4be69dbbf699f4215f8c"
format_stamp: "Formatted at 2026-08-10 07:14:43 on dist-test-slave-10pc"
server_key: "02a149ac5a3f16220e81673d74f36901"
server_key_iv: "ee403ed9a89a7279dca4dc95ade45e64"
server_key_version: "encryptionkey@0"
I20260810 07:14:43.108430 16250 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:14:43.109395 16265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:43.109695 16250 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:14:43.109802 16250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "53bbb932870a4be69dbbf699f4215f8c"
format_stamp: "Formatted at 2026-08-10 07:14:43 on dist-test-slave-10pc"
server_key: "02a149ac5a3f16220e81673d74f36901"
server_key_iv: "ee403ed9a89a7279dca4dc95ade45e64"
server_key_version: "encryptionkey@0"
I20260810 07:14:43.109889 16250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:14:43.124390 16250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:14:43.127085 16250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:14:43.127296 16250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:14:43.131930 16250 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:36347
I20260810 07:14:43.131989 16324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:36347 every 8 connection(s)
I20260810 07:14:43.132412 16250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:14:43.133422 16325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:14:43.135852 16325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53bbb932870a4be69dbbf699f4215f8c: Bootstrap starting.
I20260810 07:14:43.136606 16325 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53bbb932870a4be69dbbf699f4215f8c: Neither blocks nor log segments found. Creating new log.
I20260810 07:14:43.136941 16325 log.cc:824] T 00000000000000000000000000000000 P 53bbb932870a4be69dbbf699f4215f8c: Log is configured to *not* fsync() on all Append() calls
I20260810 07:14:43.137754 16325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53bbb932870a4be69dbbf699f4215f8c: No bootstrap required, opened a new log
I20260810 07:14:43.139413 16325 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53bbb932870a4be69dbbf699f4215f8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53bbb932870a4be69dbbf699f4215f8c" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 36347 } }
I20260810 07:14:43.139572 16325 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53bbb932870a4be69dbbf699f4215f8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:14:43.139604 16325 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53bbb932870a4be69dbbf699f4215f8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53bbb932870a4be69dbbf699f4215f8c, State: Initialized, Role: FOLLOWER
I20260810 07:14:43.139787 16325 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53bbb932870a4be69dbbf699f4215f8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53bbb932870a4be69dbbf699f4215f8c" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 36347 } }
I20260810 07:14:43.139882 16325 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53bbb932870a4be69dbbf699f4215f8c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:14:43.139946 16325 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53bbb932870a4be69dbbf699f4215f8c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:14:43.140000 16325 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53bbb932870a4be69dbbf699f4215f8c [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:14:43.140435 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 16250
I20260810 07:14:43.140578 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:14:43.140837 16325 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53bbb932870a4be69dbbf699f4215f8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53bbb932870a4be69dbbf699f4215f8c" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 36347 } }
I20260810 07:14:43.140954 16325 leader_election.cc:302] T 00000000000000000000000000000000 P 53bbb932870a4be69dbbf699f4215f8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53bbb932870a4be69dbbf699f4215f8c; no voters:
I20260810 07:14:43.141113 16325 leader_election.cc:288] T 00000000000000000000000000000000 P 53bbb932870a4be69dbbf699f4215f8c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:14:43.141182 16328 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53bbb932870a4be69dbbf699f4215f8c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:14:43.141405 16229 external_mini_cluster.cc:1428] Setting key 288b638670153c0824ab4d175ed9432b
I20260810 07:14:43.141387 16328 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53bbb932870a4be69dbbf699f4215f8c [term 1 LEADER]: Becoming Leader. State: Replica: 53bbb932870a4be69dbbf699f4215f8c, State: Running, Role: LEADER
I20260810 07:14:43.141398 16325 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53bbb932870a4be69dbbf699f4215f8c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:14:43.141537 16328 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53bbb932870a4be69dbbf699f4215f8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53bbb932870a4be69dbbf699f4215f8c" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 36347 } }
I20260810 07:14:43.141955 16328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53bbb932870a4be69dbbf699f4215f8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53bbb932870a4be69dbbf699f4215f8c. Latest consensus state: current_term: 1 leader_uuid: "53bbb932870a4be69dbbf699f4215f8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53bbb932870a4be69dbbf699f4215f8c" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 36347 } } }
I20260810 07:14:43.142045 16328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53bbb932870a4be69dbbf699f4215f8c [sys.catalog]: This master's current role is: LEADER
I20260810 07:14:43.142115 16329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53bbb932870a4be69dbbf699f4215f8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53bbb932870a4be69dbbf699f4215f8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53bbb932870a4be69dbbf699f4215f8c" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 36347 } } }
I20260810 07:14:43.142181 16329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53bbb932870a4be69dbbf699f4215f8c [sys.catalog]: This master's current role is: LEADER
I20260810 07:14:43.142266 16333 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:14:43.142863 16333 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:14:43.147116 16333 catalog_manager.cc:1347] Generated new cluster ID: 8366036d93a64bdeaaf53fef5503e5d0
I20260810 07:14:43.147176 16333 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 07:14:43 dist-test-slave-10pc krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:14:43.160205 16346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:43.142992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51240 (local address 127.15.217.126:36347)
0810 07:14:43.143252 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:43.143261 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:43.143582 (+ 321us) server_negotiation.cc:407] Connection header received
0810 07:14:43.145947 (+ 2365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:43.145953 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:43.146005 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:43.146167 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:43.148502 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:43.149457 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:43.150688 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:43.151115 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:43.151241 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:43.155497 (+ 4256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:43.155502 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:43.155503 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:43.155516 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:43.157908 (+ 2392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:43.158404 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:43.158408 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:43.158408 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:43.158514 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:43.158804 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:43.158805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:43.158805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:43.159197 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:43.159360 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:43.159673 (+ 313us) server_negotiation.cc:299] Negotiation successful
0810 07:14:43.159830 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":86,"threads_started":1}
I20260810 07:14:43.163084 16333 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:14:43.163616 16333 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:14:43.166392 16333 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53bbb932870a4be69dbbf699f4215f8c: Generated new TSK 0
I20260810 07:14:43.166596 16333 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab.
Entry for principal kudu/127.15.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab.
I20260810 07:14:43.203331 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.65:0
--local_ip_for_outbound_sockets=127.15.217.65
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:36347
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:14:43.322312 16352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:14:43.322564 16352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:14:43.322593 16352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:14:43.323958 16352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:14:43.324010 16352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:14:43.324041 16352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.65
I20260810 07:14:43.326068 16352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:36347
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.65
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:14:43.326385 16352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:14:43.326670 16352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:14:43.332976 16361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:43.333035 16357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:43.333079 16359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:14:43.333488 16352 server_base.cc:1034] running on GCE node
I20260810 07:14:43.333725 16352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:14:43.334014 16352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:14:43.335194 16352 hybrid_clock.cc:648] HybridClock initialized: now 1786346083335171 us; error 42 us; skew 500 ppm
Aug 10 07:14:43 dist-test-slave-10pc krb5kdc[16236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346083, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:14:43.338433 16352 init.cc:377] Logged in from keytab as kudu/127.15.217.65@KRBTEST.COM (short username kudu)
I20260810 07:14:43.338914 16352 webserver.cc:460] Webserver started at http://127.15.217.65:40407/ using document root <none> and password file <none>
I20260810 07:14:43.339170 16352 fs_manager.cc:359] Metadata directory not provided
I20260810 07:14:43.339246 16352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:14:43.339397 16352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:14:43.340744 16352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-0/data/instance:
uuid: "fc77555d8d7e45ba9de1782c65e02197"
format_stamp: "Formatted at 2026-08-10 07:14:43 on dist-test-slave-10pc"
server_key: "15e6416ec233613b86dc90e59158bd9f"
server_key_iv: "2bf1c67f695b8aa31a709000c765354c"
server_key_version: "encryptionkey@0"
I20260810 07:14:43.341251 16352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance:
uuid: "fc77555d8d7e45ba9de1782c65e02197"
format_stamp: "Formatted at 2026-08-10 07:14:43 on dist-test-slave-10pc"
server_key: "15e6416ec233613b86dc90e59158bd9f"
server_key_iv: "2bf1c67f695b8aa31a709000c765354c"
server_key_version: "encryptionkey@0"
I20260810 07:14:43.343040 16352 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:14:43.344120 16369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:43.344321 16352 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:14:43.344434 16352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-0/wal
uuid: "fc77555d8d7e45ba9de1782c65e02197"
format_stamp: "Formatted at 2026-08-10 07:14:43 on dist-test-slave-10pc"
server_key: "15e6416ec233613b86dc90e59158bd9f"
server_key_iv: "2bf1c67f695b8aa31a709000c765354c"
server_key_version: "encryptionkey@0"
I20260810 07:14:43.344571 16352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:14:43.357419 16352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:14:43.359879 16352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:14:43.360110 16352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:14:43.360702 16352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:14:43.361104 16352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:14:43.361163 16352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:43.361200 16352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:14:43.361250 16352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:43.376802 16352 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.65:35997
I20260810 07:14:43.377020 16493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.65:35997 every 8 connection(s)
I20260810 07:14:43.377345 16352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
I20260810 07:14:43.381079 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 16352
I20260810 07:14:43.381204 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance
I20260810 07:14:43.381387 16229 external_mini_cluster.cc:1428] Setting key 3fcc6b44e8194b11acf6bacfbb7297b5
Aug 10 07:14:43 dist-test-slave-10pc krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346083, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:14:43.390502 16346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:43.378674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:58183 (local address 127.15.217.126:36347)
0810 07:14:43.378750 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:43.378752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:43.379121 (+ 369us) server_negotiation.cc:407] Connection header received
0810 07:14:43.379967 (+ 846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:43.379969 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:43.379981 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:43.380082 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:43.381874 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:43.382482 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:43.383691 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:43.383902 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:43.383983 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:43.387026 (+ 3043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:43.387027 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:43.387027 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:43.387035 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:43.388757 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:43.389327 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:43.389328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:43.389328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:43.389389 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:43.389718 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:43.389719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:43.389719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:43.389833 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:43.389886 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:43.390333 (+ 447us) server_negotiation.cc:299] Negotiation successful
0810 07:14:43.390415 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 07:14:43.390924 16496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:43.378647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:36347 (local address 127.15.217.65:58183)
0810 07:14:43.378937 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:43.378986 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:43.379852 (+ 866us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:43.380203 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:43.380207 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:43.381317 (+ 1110us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:14:43.381766 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:43.381778 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:43.382585 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:43.382586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:43.383592 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:43.383595 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:43.383995 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:43.383996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:43.384192 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:43.384874 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:14:43.384883 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:14:43.386905 (+ 2022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:14:43.388844 (+ 1939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:43.388853 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:43.388856 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:43.389234 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:43.389467 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:43.389468 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:43.389468 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:43.389645 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:43.389916 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:14:43.389933 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:14:43.390136 (+ 203us) client_negotiation.cc:769] Sending connection context
0810 07:14:43.390303 (+ 167us) client_negotiation.cc:240] Negotiation successful
0810 07:14:43.390594 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":140,"threads_started":1}
I20260810 07:14:43.391860 16494 heartbeater.cc:347] Connected to a master server at 127.15.217.126:36347
I20260810 07:14:43.391985 16494 heartbeater.cc:464] Registering TS with master...
I20260810 07:14:43.392264 16494 heartbeater.cc:511] Master 127.15.217.126:36347 requested a full tablet report, sending...
I20260810 07:14:43.392927 16284 ts_manager.cc:194] Registered new tserver with Master: fc77555d8d7e45ba9de1782c65e02197 (127.15.217.65:35997)
WARNING: no policy specified for kudu/127.15.217.66@KRBTEST.COM; defaulting to no policy
I20260810 07:14:43.393581 16284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.65@KRBTEST.COM'} at 127.15.217.65:58183
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab.
Entry for principal kudu/127.15.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab.
I20260810 07:14:43.415282 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.66:0
--local_ip_for_outbound_sockets=127.15.217.66
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:36347
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:14:43.525849 16500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:14:43.526129 16500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:14:43.526175 16500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:14:43.527489 16500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:14:43.527558 16500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:14:43.527606 16500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.66
I20260810 07:14:43.529422 16500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:36347
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.66
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:14:43.529735 16500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:14:43.530006 16500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:14:43.536979 16505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:43.537097 16506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:43.537133 16508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:14:43.537415 16500 server_base.cc:1034] running on GCE node
I20260810 07:14:43.537672 16500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:14:43.537931 16500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:14:43.539077 16500 hybrid_clock.cc:648] HybridClock initialized: now 1786346083539058 us; error 39 us; skew 500 ppm
Aug 10 07:14:43 dist-test-slave-10pc krb5kdc[16236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346083, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:14:43.542587 16500 init.cc:377] Logged in from keytab as kudu/127.15.217.66@KRBTEST.COM (short username kudu)
I20260810 07:14:43.543092 16500 webserver.cc:460] Webserver started at http://127.15.217.66:35503/ using document root <none> and password file <none>
I20260810 07:14:43.543385 16500 fs_manager.cc:359] Metadata directory not provided
I20260810 07:14:43.543434 16500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:14:43.543530 16500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:14:43.544848 16500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-1/data/instance:
uuid: "34d5f4e137d243b69d30394c3aebb3bd"
format_stamp: "Formatted at 2026-08-10 07:14:43 on dist-test-slave-10pc"
server_key: "d7a2b3955badaa757b3431d9685c053e"
server_key_iv: "43deb3a234c2203dbef3a65189af992e"
server_key_version: "encryptionkey@0"
I20260810 07:14:43.545233 16500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance:
uuid: "34d5f4e137d243b69d30394c3aebb3bd"
format_stamp: "Formatted at 2026-08-10 07:14:43 on dist-test-slave-10pc"
server_key: "d7a2b3955badaa757b3431d9685c053e"
server_key_iv: "43deb3a234c2203dbef3a65189af992e"
server_key_version: "encryptionkey@0"
I20260810 07:14:43.546825 16500 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260810 07:14:43.547724 16516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:43.547927 16500 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:14:43.548029 16500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-1/wal
uuid: "34d5f4e137d243b69d30394c3aebb3bd"
format_stamp: "Formatted at 2026-08-10 07:14:43 on dist-test-slave-10pc"
server_key: "d7a2b3955badaa757b3431d9685c053e"
server_key_iv: "43deb3a234c2203dbef3a65189af992e"
server_key_version: "encryptionkey@0"
I20260810 07:14:43.548125 16500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:14:43.557410 16500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:14:43.559612 16500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:14:43.559839 16500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:14:43.560240 16500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:14:43.560726 16500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:14:43.560786 16500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:43.560830 16500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:14:43.560858 16500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:43.574887 16500 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.66:44513
I20260810 07:14:43.574985 16646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.66:44513 every 8 connection(s)
I20260810 07:14:43.575363 16500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
I20260810 07:14:43.582840 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 16500
I20260810 07:14:43.582958 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance
Aug 10 07:14:43 dist-test-slave-10pc krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346083, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:14:43.583194 16229 external_mini_cluster.cc:1428] Setting key fd8899bf7187805f511e1bf342762f14
I20260810 07:14:43.587188 16346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:43.576245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:46307 (local address 127.15.217.126:36347)
0810 07:14:43.576293 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:43.576295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:43.576764 (+ 469us) server_negotiation.cc:407] Connection header received
0810 07:14:43.577642 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:43.577643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:43.577657 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:43.577713 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:43.578908 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:43.579415 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:43.580684 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:43.580889 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:43.580965 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:43.583680 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:43.583681 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:43.583682 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:43.583690 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:43.585281 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:43.585818 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:43.585819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:43.585819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:43.585923 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:43.586416 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:43.586417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:43.586418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:43.586608 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:43.586698 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:43.586963 (+ 265us) server_negotiation.cc:299] Negotiation successful
0810 07:14:43.587065 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 07:14:43.587291 16649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:43.576233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:36347 (local address 127.15.217.66:46307)
0810 07:14:43.576610 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:43.576650 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:43.577497 (+ 847us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:43.577807 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:43.577812 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:43.578460 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:14:43.578782 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:43.578788 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:43.579524 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:43.579526 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:43.580592 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:43.580595 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:43.580987 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:43.580989 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:43.581189 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:43.581793 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:14:43.581803 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:14:43.583560 (+ 1757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:14:43.585367 (+ 1807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:43.585375 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:43.585378 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:43.585717 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:43.586025 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:43.586027 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:43.586027 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:43.586307 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:43.586689 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:14:43.586695 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:14:43.586822 (+ 127us) client_negotiation.cc:769] Sending connection context
0810 07:14:43.586914 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 07:14:43.587041 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":180,"threads_started":1}
I20260810 07:14:43.587831 16647 heartbeater.cc:347] Connected to a master server at 127.15.217.126:36347
I20260810 07:14:43.587951 16647 heartbeater.cc:464] Registering TS with master...
I20260810 07:14:43.588162 16647 heartbeater.cc:511] Master 127.15.217.126:36347 requested a full tablet report, sending...
I20260810 07:14:43.588634 16284 ts_manager.cc:194] Registered new tserver with Master: 34d5f4e137d243b69d30394c3aebb3bd (127.15.217.66:44513)
I20260810 07:14:43.589157 16284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.66@KRBTEST.COM'} at 127.15.217.66:46307
WARNING: no policy specified for kudu/127.15.217.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab.
Entry for principal kudu/127.15.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab.
I20260810 07:14:43.615587 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.67:0
--local_ip_for_outbound_sockets=127.15.217.67
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:36347
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:14:43.742777 16652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:14:43.743039 16652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:14:43.743088 16652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:14:43.744572 16652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:14:43.744648 16652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:14:43.744699 16652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.67
I20260810 07:14:43.746428 16652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:36347
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.67
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:14:43.746737 16652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:14:43.746994 16652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:14:43.753774 16661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:43.753866 16659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:43.753850 16658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:14:43.754240 16652 server_base.cc:1034] running on GCE node
I20260810 07:14:43.754484 16652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:14:43.754710 16652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:14:43.755901 16652 hybrid_clock.cc:648] HybridClock initialized: now 1786346083755881 us; error 27 us; skew 500 ppm
Aug 10 07:14:43 dist-test-slave-10pc krb5kdc[16236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346083, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:14:43.758814 16652 init.cc:377] Logged in from keytab as kudu/127.15.217.67@KRBTEST.COM (short username kudu)
I20260810 07:14:43.759289 16652 webserver.cc:460] Webserver started at http://127.15.217.67:45509/ using document root <none> and password file <none>
I20260810 07:14:43.759533 16652 fs_manager.cc:359] Metadata directory not provided
I20260810 07:14:43.759613 16652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:14:43.759769 16652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:14:43.760982 16652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-2/data/instance:
uuid: "f6ceda3590bf444085585e6ded06e499"
format_stamp: "Formatted at 2026-08-10 07:14:43 on dist-test-slave-10pc"
server_key: "406af754a40740e2fb7549a2e88b7de3"
server_key_iv: "9cafdb21ff466a13f0c0dc180506af75"
server_key_version: "encryptionkey@0"
I20260810 07:14:43.761374 16652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance:
uuid: "f6ceda3590bf444085585e6ded06e499"
format_stamp: "Formatted at 2026-08-10 07:14:43 on dist-test-slave-10pc"
server_key: "406af754a40740e2fb7549a2e88b7de3"
server_key_iv: "9cafdb21ff466a13f0c0dc180506af75"
server_key_version: "encryptionkey@0"
I20260810 07:14:43.763129 16652 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:14:43.763995 16668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:43.764194 16652 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:14:43.764302 16652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-2/wal
uuid: "f6ceda3590bf444085585e6ded06e499"
format_stamp: "Formatted at 2026-08-10 07:14:43 on dist-test-slave-10pc"
server_key: "406af754a40740e2fb7549a2e88b7de3"
server_key_iv: "9cafdb21ff466a13f0c0dc180506af75"
server_key_version: "encryptionkey@0"
I20260810 07:14:43.764384 16652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:14:43.785061 16652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:14:43.787639 16652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:14:43.787820 16652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:14:43.788095 16652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:14:43.788467 16652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:14:43.788556 16652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:43.788611 16652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:14:43.788645 16652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:43.803683 16652 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.67:42009
I20260810 07:14:43.803854 16792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.67:42009 every 8 connection(s)
I20260810 07:14:43.804181 16652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
Aug 10 07:14:43 dist-test-slave-10pc krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346083, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:14:43.812652 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 16652
I20260810 07:14:43.812762 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance
I20260810 07:14:43.812954 16229 external_mini_cluster.cc:1428] Setting key 6a40dd7e8e2d6ac8d15f6388c2a157c9
I20260810 07:14:43.815339 16346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:43.805147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:45379 (local address 127.15.217.126:36347)
0810 07:14:43.805211 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:43.805213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:43.805553 (+ 340us) server_negotiation.cc:407] Connection header received
0810 07:14:43.806312 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:43.806313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:43.806326 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:43.806385 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:43.807499 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:43.808020 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:43.809267 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:43.809470 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:43.809552 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:43.812170 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:43.812172 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:43.812172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:43.812182 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:43.814047 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:43.814471 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:43.814472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:43.814472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:43.814530 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:43.814760 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:43.814761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:43.814761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:43.814873 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:43.814926 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:43.815205 (+ 279us) server_negotiation.cc:299] Negotiation successful
0810 07:14:43.815274 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 07:14:43.815574 16796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:43.805129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:36347 (local address 127.15.217.67:45379)
0810 07:14:43.805401 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:43.805442 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:43.806192 (+ 750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:43.806467 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:43.806471 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:43.807126 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:14:43.807388 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:43.807393 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:43.808137 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:43.808138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:43.809159 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:43.809170 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:43.809562 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:43.809563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:43.809751 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:43.810353 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:14:43.810362 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:14:43.812063 (+ 1701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:14:43.814132 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:43.814139 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:43.814142 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:43.814399 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:43.814596 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:43.814597 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:43.814597 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:43.814701 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:43.814931 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:14:43.814936 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:14:43.815062 (+ 126us) client_negotiation.cc:769] Sending connection context
0810 07:14:43.815164 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 07:14:43.815294 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":129,"threads_started":1}
I20260810 07:14:43.816144 16793 heartbeater.cc:347] Connected to a master server at 127.15.217.126:36347
I20260810 07:14:43.816257 16793 heartbeater.cc:464] Registering TS with master...
I20260810 07:14:43.816493 16793 heartbeater.cc:511] Master 127.15.217.126:36347 requested a full tablet report, sending...
I20260810 07:14:43.816905 16284 ts_manager.cc:194] Registered new tserver with Master: f6ceda3590bf444085585e6ded06e499 (127.15.217.67:42009)
I20260810 07:14:43.817294 16284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.67@KRBTEST.COM'} at 127.15.217.67:45379
I20260810 07:14:43.825220 16229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 07:14:43 dist-test-slave-10pc krb5kdc[16236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346083, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 07:14:43 dist-test-slave-10pc krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346083, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.217.65@KRBTEST.COM
I20260810 07:14:43.854542 16801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:43.845160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33884 (local address 127.15.217.65:35997)
0810 07:14:43.845339 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:43.845346 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:43.845359 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:14:43.845396 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:43.845398 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:43.845413 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:43.845506 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:43.846215 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:43.847663 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:43.848595 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:43.849103 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:43.849166 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:43.851522 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:43.851531 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:43.851532 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:43.851539 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:43.853367 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:43.853710 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:43.853713 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:43.853714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:43.853772 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:43.853943 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:43.853944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:43.853944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:43.854081 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:43.854203 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:43.854276 (+ 73us) server_negotiation.cc:299] Negotiation successful
0810 07:14:43.854343 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":71,"threads_started":1}
W20260810 07:14:43.855057 16399 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33884
Aug 10 07:14:43 dist-test-slave-10pc krb5kdc[16236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346083, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 07:14:43 dist-test-slave-10pc krb5kdc[16236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346083, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.65@KRBTEST.COM
I20260810 07:14:43.880851 16801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:43.873185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33890 (local address 127.15.217.65:35997)
0810 07:14:43.873255 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:43.873257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:43.873313 (+ 56us) server_negotiation.cc:407] Connection header received
0810 07:14:43.873371 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:43.873372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:43.873389 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:43.873436 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:43.873990 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:43.874623 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:43.875417 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:43.875675 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:43.875729 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:43.878252 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:43.878254 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:43.878255 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:43.878264 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:43.879807 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:43.880134 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:43.880134 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:43.880135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:43.880190 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:43.880358 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:43.880358 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:43.880359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:43.880490 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:43.880679 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:43.880711 (+ 32us) server_negotiation.cc:299] Negotiation successful
0810 07:14:43.880779 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 07:14:43.881489 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 16352
I20260810 07:14:43.890631 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 16500
I20260810 07:14:43.898396 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 16652
I20260810 07:14:43.905236 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 16250
2026-08-10T07:14:43Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2156 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:14:43 dist-test-slave-10pc krb5kdc[16809](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:14:43 dist-test-slave-10pc krb5kdc[16809](info): set up 2 sockets
Aug 10 07:14:43 dist-test-slave-10pc krb5kdc[16809](info): commencing operation
krb5kdc: starting...
W20260810 07:14:45.062638 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:14:45 dist-test-slave-10pc krb5kdc[16809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:14:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:14:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
I20260810 07:14:45.190658 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:38383
--webserver_interface=127.15.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:45145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:38383
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:14:45.277508 16824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:14:45.277788 16824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:14:45.277830 16824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:14:45.279062 16824 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:14:45.279137 16824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:14:45.279160 16824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:14:45.279178 16824 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:14:45.279199 16824 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:14:45.280750 16824 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:45145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:38383
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:38383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:14:45.281059 16824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:14:45.281307 16824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:14:45.286638 16824 server_base.cc:1034] running on GCE node
W20260810 07:14:45.286602 16831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:45.286546 16833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:45.286546 16830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:14:45.286995 16824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:14:45.287269 16824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:14:45.288492 16824 hybrid_clock.cc:648] HybridClock initialized: now 1786346085288435 us; error 45 us; skew 500 ppm
Aug 10 07:14:45 dist-test-slave-10pc krb5kdc[16809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346085, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:14:45.291258 16824 init.cc:377] Logged in from keytab as kudu/127.15.217.126@KRBTEST.COM (short username kudu)
I20260810 07:14:45.291743 16824 webserver.cc:460] Webserver started at http://127.15.217.126:46061/ using document root <none> and password file <none>
I20260810 07:14:45.291939 16824 fs_manager.cc:359] Metadata directory not provided
I20260810 07:14:45.291981 16824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:14:45.292074 16824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:14:45.293257 16824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "d915df809ed7430b81a211a0e3b157cd"
format_stamp: "Formatted at 2026-08-10 07:14:45 on dist-test-slave-10pc"
server_key: "04a1bee779f80d719ba2adee4a03791d"
server_key_iv: "3c061935b65afd6555d2ee7179485bb1"
server_key_version: "encryptionkey@0"
I20260810 07:14:45.293587 16824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "d915df809ed7430b81a211a0e3b157cd"
format_stamp: "Formatted at 2026-08-10 07:14:45 on dist-test-slave-10pc"
server_key: "04a1bee779f80d719ba2adee4a03791d"
server_key_iv: "3c061935b65afd6555d2ee7179485bb1"
server_key_version: "encryptionkey@0"
I20260810 07:14:45.294914 16824 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:14:45.295660 16841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:45.295853 16824 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:14:45.295955 16824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "d915df809ed7430b81a211a0e3b157cd"
format_stamp: "Formatted at 2026-08-10 07:14:45 on dist-test-slave-10pc"
server_key: "04a1bee779f80d719ba2adee4a03791d"
server_key_iv: "3c061935b65afd6555d2ee7179485bb1"
server_key_version: "encryptionkey@0"
I20260810 07:14:45.296038 16824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:14:45.307834 16824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:14:45.309875 16824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:14:45.310060 16824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:14:45.314529 16824 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:38383
I20260810 07:14:45.314577 16896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:38383 every 8 connection(s)
I20260810 07:14:45.314985 16824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:14:45.315915 16897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:14:45.316092 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 16824
I20260810 07:14:45.316210 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:14:45.316388 16229 external_mini_cluster.cc:1428] Setting key 2e8b94cd53d2275bb18887c460295337
I20260810 07:14:45.319667 16897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d915df809ed7430b81a211a0e3b157cd: Bootstrap starting.
I20260810 07:14:45.320380 16897 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d915df809ed7430b81a211a0e3b157cd: Neither blocks nor log segments found. Creating new log.
I20260810 07:14:45.320721 16897 log.cc:824] T 00000000000000000000000000000000 P d915df809ed7430b81a211a0e3b157cd: Log is configured to *not* fsync() on all Append() calls
I20260810 07:14:45.321466 16897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d915df809ed7430b81a211a0e3b157cd: No bootstrap required, opened a new log
Aug 10 07:14:45 dist-test-slave-10pc krb5kdc[16809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:14:45.323652 16897 raft_consensus.cc:348] T 00000000000000000000000000000000 P d915df809ed7430b81a211a0e3b157cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d915df809ed7430b81a211a0e3b157cd" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 38383 } }
I20260810 07:14:45.323813 16897 raft_consensus.cc:374] T 00000000000000000000000000000000 P d915df809ed7430b81a211a0e3b157cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:14:45.323864 16897 raft_consensus.cc:729] T 00000000000000000000000000000000 P d915df809ed7430b81a211a0e3b157cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d915df809ed7430b81a211a0e3b157cd, State: Initialized, Role: FOLLOWER
I20260810 07:14:45.324033 16897 consensus_queue.cc:260] T 00000000000000000000000000000000 P d915df809ed7430b81a211a0e3b157cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d915df809ed7430b81a211a0e3b157cd" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 38383 } }
I20260810 07:14:45.324143 16897 raft_consensus.cc:388] T 00000000000000000000000000000000 P d915df809ed7430b81a211a0e3b157cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:14:45.324180 16897 raft_consensus.cc:482] T 00000000000000000000000000000000 P d915df809ed7430b81a211a0e3b157cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:14:45.324239 16897 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d915df809ed7430b81a211a0e3b157cd [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:14:45.325042 16897 raft_consensus.cc:504] T 00000000000000000000000000000000 P d915df809ed7430b81a211a0e3b157cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d915df809ed7430b81a211a0e3b157cd" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 38383 } }
I20260810 07:14:45.325179 16897 leader_election.cc:302] T 00000000000000000000000000000000 P d915df809ed7430b81a211a0e3b157cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d915df809ed7430b81a211a0e3b157cd; no voters:
I20260810 07:14:45.325354 16897 leader_election.cc:288] T 00000000000000000000000000000000 P d915df809ed7430b81a211a0e3b157cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:14:45.325431 16902 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d915df809ed7430b81a211a0e3b157cd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:14:45.325626 16902 raft_consensus.cc:686] T 00000000000000000000000000000000 P d915df809ed7430b81a211a0e3b157cd [term 1 LEADER]: Becoming Leader. State: Replica: d915df809ed7430b81a211a0e3b157cd, State: Running, Role: LEADER
I20260810 07:14:45.325639 16897 sys_catalog.cc:564] T 00000000000000000000000000000000 P d915df809ed7430b81a211a0e3b157cd [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:14:45.325821 16902 consensus_queue.cc:237] T 00000000000000000000000000000000 P d915df809ed7430b81a211a0e3b157cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d915df809ed7430b81a211a0e3b157cd" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 38383 } }
I20260810 07:14:45.326697 16905 sys_catalog.cc:455] T 00000000000000000000000000000000 P d915df809ed7430b81a211a0e3b157cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader d915df809ed7430b81a211a0e3b157cd. Latest consensus state: current_term: 1 leader_uuid: "d915df809ed7430b81a211a0e3b157cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d915df809ed7430b81a211a0e3b157cd" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 38383 } } }
I20260810 07:14:45.326784 16905 sys_catalog.cc:458] T 00000000000000000000000000000000 P d915df809ed7430b81a211a0e3b157cd [sys.catalog]: This master's current role is: LEADER
I20260810 07:14:45.327069 16914 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:14:45.327471 16903 sys_catalog.cc:455] T 00000000000000000000000000000000 P d915df809ed7430b81a211a0e3b157cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d915df809ed7430b81a211a0e3b157cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d915df809ed7430b81a211a0e3b157cd" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 38383 } } }
I20260810 07:14:45.327558 16903 sys_catalog.cc:458] T 00000000000000000000000000000000 P d915df809ed7430b81a211a0e3b157cd [sys.catalog]: This master's current role is: LEADER
I20260810 07:14:45.327636 16914 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:14:45.328642 16900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:45.317001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60276 (local address 127.15.217.126:38383)
0810 07:14:45.317206 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:45.317216 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:45.317252 (+ 36us) server_negotiation.cc:407] Connection header received
0810 07:14:45.317792 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:45.317802 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:45.317881 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:45.318042 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:45.318637 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:45.320028 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:45.320888 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:45.321464 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:45.321595 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:45.323953 (+ 2358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:45.323957 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:45.323957 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:45.323970 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:45.326311 (+ 2341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:45.326777 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:45.326780 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:45.326780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:45.326920 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:45.327180 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:45.327180 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:45.327181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:45.327504 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:45.328175 (+ 671us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:45.328250 (+ 75us) server_negotiation.cc:299] Negotiation successful
0810 07:14:45.328402 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":95,"threads_started":1}
I20260810 07:14:45.329707 16914 catalog_manager.cc:1347] Generated new cluster ID: f9a0dc1dc6374d7ca3705c192e675e85
I20260810 07:14:45.329762 16914 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:14:45.338151 16914 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:14:45.338723 16914 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:14:45.341976 16914 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d915df809ed7430b81a211a0e3b157cd: Generated new TSK 0
I20260810 07:14:45.342263 16914 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab.
Entry for principal kudu/127.15.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab.
I20260810 07:14:45.375927 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.65:0
--local_ip_for_outbound_sockets=127.15.217.65
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:38383
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:45145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:14:45.463369 16925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:14:45.463619 16925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:14:45.463660 16925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:14:45.464862 16925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:14:45.464947 16925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:14:45.465000 16925 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.65
I20260810 07:14:45.466552 16925 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:45145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:38383
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.65
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:14:45.466856 16925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:14:45.467154 16925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:14:45.472575 16933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:45.472589 16930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:45.472592 16931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:14:45.472846 16925 server_base.cc:1034] running on GCE node
I20260810 07:14:45.473148 16925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:14:45.473412 16925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:14:45.474577 16925 hybrid_clock.cc:648] HybridClock initialized: now 1786346085474580 us; error 46 us; skew 500 ppm
Aug 10 07:14:45 dist-test-slave-10pc krb5kdc[16809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346085, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:14:45.477356 16925 init.cc:377] Logged in from keytab as kudu/127.15.217.65@KRBTEST.COM (short username kudu)
I20260810 07:14:45.477881 16925 webserver.cc:460] Webserver started at http://127.15.217.65:46535/ using document root <none> and password file <none>
I20260810 07:14:45.478130 16925 fs_manager.cc:359] Metadata directory not provided
I20260810 07:14:45.478201 16925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:14:45.478359 16925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:14:45.479524 16925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-0/data/instance:
uuid: "529b70e09ebb412d8b2f0fa4303eb4f8"
format_stamp: "Formatted at 2026-08-10 07:14:45 on dist-test-slave-10pc"
server_key: "c2a6e07d9091b983f41c1bf986152003"
server_key_iv: "7417ac5beb121f9e95ea47cc235c299c"
server_key_version: "encryptionkey@0"
I20260810 07:14:45.479939 16925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance:
uuid: "529b70e09ebb412d8b2f0fa4303eb4f8"
format_stamp: "Formatted at 2026-08-10 07:14:45 on dist-test-slave-10pc"
server_key: "c2a6e07d9091b983f41c1bf986152003"
server_key_iv: "7417ac5beb121f9e95ea47cc235c299c"
server_key_version: "encryptionkey@0"
I20260810 07:14:45.481451 16925 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:14:45.482213 16942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:45.482429 16925 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:14:45.482570 16925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-0/wal
uuid: "529b70e09ebb412d8b2f0fa4303eb4f8"
format_stamp: "Formatted at 2026-08-10 07:14:45 on dist-test-slave-10pc"
server_key: "c2a6e07d9091b983f41c1bf986152003"
server_key_iv: "7417ac5beb121f9e95ea47cc235c299c"
server_key_version: "encryptionkey@0"
I20260810 07:14:45.482656 16925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:14:45.499864 16925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:14:45.502140 16925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:14:45.502314 16925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:14:45.502584 16925 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:14:45.502928 16925 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:14:45.502987 16925 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:45.503038 16925 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:14:45.503070 16925 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:45.509497 16925 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.65:41959
I20260810 07:14:45.509550 17067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.65:41959 every 8 connection(s)
I20260810 07:14:45.509907 16925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
I20260810 07:14:45.512599 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 16925
I20260810 07:14:45.512681 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance
I20260810 07:14:45.512872 16229 external_mini_cluster.cc:1428] Setting key e88cca57babb93a9de3631d3ac3f0a29
Aug 10 07:14:45 dist-test-slave-10pc krb5kdc[16809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346085, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:14:45.521638 16900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:45.510714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:54887 (local address 127.15.217.126:38383)
0810 07:14:45.510784 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:45.510787 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:45.510979 (+ 192us) server_negotiation.cc:407] Connection header received
0810 07:14:45.512125 (+ 1146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:45.512127 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:45.512140 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:45.512214 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:45.513170 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:45.513788 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:45.515422 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:45.515772 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:45.515853 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:45.518332 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:45.518334 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:45.518335 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:45.518345 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:45.520099 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:45.520579 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:45.520581 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:45.520581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:45.520652 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:45.520922 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:45.520923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:45.520923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:45.521047 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:45.521134 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:45.521427 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 07:14:45.521525 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 07:14:45.521811 17070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:45.510646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:38383 (local address 127.15.217.65:54887)
0810 07:14:45.510831 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:45.510877 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:45.511991 (+ 1114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:45.512305 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:45.512309 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:45.512767 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:14:45.513059 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:45.513066 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:45.513910 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:45.513911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:45.515307 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:45.515311 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:45.515875 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:45.515875 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:45.515990 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:45.516621 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:14:45.516630 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:14:45.518223 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:14:45.520197 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:45.520204 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:45.520207 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:45.520434 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:45.520727 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:45.520728 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:45.520728 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:45.520844 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:45.521138 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:14:45.521146 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:14:45.521286 (+ 140us) client_negotiation.cc:769] Sending connection context
0810 07:14:45.521390 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 07:14:45.521558 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":101,"threads_started":1}
I20260810 07:14:45.522375 17068 heartbeater.cc:347] Connected to a master server at 127.15.217.126:38383
I20260810 07:14:45.522505 17068 heartbeater.cc:464] Registering TS with master...
I20260810 07:14:45.522743 17068 heartbeater.cc:511] Master 127.15.217.126:38383 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.15.217.66@KRBTEST.COM; defaulting to no policy
I20260810 07:14:45.523358 16859 ts_manager.cc:194] Registered new tserver with Master: 529b70e09ebb412d8b2f0fa4303eb4f8 (127.15.217.65:41959)
I20260810 07:14:45.523986 16859 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.65@KRBTEST.COM'} at 127.15.217.65:54887
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab.
Entry for principal kudu/127.15.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab.
I20260810 07:14:45.543172 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.66:0
--local_ip_for_outbound_sockets=127.15.217.66
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:38383
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:45145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:14:45.630765 17073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:14:45.631011 17073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:14:45.631052 17073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:14:45.632228 17073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:14:45.632297 17073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:14:45.632342 17073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.66
I20260810 07:14:45.633917 17073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:45145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:38383
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.66
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:14:45.634210 17073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:14:45.634456 17073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:14:45.639686 17079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:45.639686 17082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:45.639709 17078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:14:45.639885 17073 server_base.cc:1034] running on GCE node
I20260810 07:14:45.640143 17073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:14:45.640347 17073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:14:45.641500 17073 hybrid_clock.cc:648] HybridClock initialized: now 1786346085641503 us; error 45 us; skew 500 ppm
Aug 10 07:14:45 dist-test-slave-10pc krb5kdc[16809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346085, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:14:45.644132 17073 init.cc:377] Logged in from keytab as kudu/127.15.217.66@KRBTEST.COM (short username kudu)
I20260810 07:14:45.644608 17073 webserver.cc:460] Webserver started at http://127.15.217.66:37303/ using document root <none> and password file <none>
I20260810 07:14:45.644804 17073 fs_manager.cc:359] Metadata directory not provided
I20260810 07:14:45.644847 17073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:14:45.644953 17073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:14:45.646029 17073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-1/data/instance:
uuid: "9e83fdfc9f3a4f0e8e4e522f561fea24"
format_stamp: "Formatted at 2026-08-10 07:14:45 on dist-test-slave-10pc"
server_key: "515934c3b6ea4dc136bbe7fef4098610"
server_key_iv: "393b2470ecff9d25584cdb6da8ce7463"
server_key_version: "encryptionkey@0"
I20260810 07:14:45.646387 17073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance:
uuid: "9e83fdfc9f3a4f0e8e4e522f561fea24"
format_stamp: "Formatted at 2026-08-10 07:14:45 on dist-test-slave-10pc"
server_key: "515934c3b6ea4dc136bbe7fef4098610"
server_key_iv: "393b2470ecff9d25584cdb6da8ce7463"
server_key_version: "encryptionkey@0"
I20260810 07:14:45.647714 17073 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:14:45.648433 17090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:45.648697 17073 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:14:45.648828 17073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-1/wal
uuid: "9e83fdfc9f3a4f0e8e4e522f561fea24"
format_stamp: "Formatted at 2026-08-10 07:14:45 on dist-test-slave-10pc"
server_key: "515934c3b6ea4dc136bbe7fef4098610"
server_key_iv: "393b2470ecff9d25584cdb6da8ce7463"
server_key_version: "encryptionkey@0"
I20260810 07:14:45.648905 17073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:14:45.661969 17073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:14:45.664122 17073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:14:45.664311 17073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:14:45.664608 17073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:14:45.664999 17073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:14:45.665060 17073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:45.665105 17073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:14:45.665132 17073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:45.671237 17073 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.66:43607
I20260810 07:14:45.671330 17218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.66:43607 every 8 connection(s)
I20260810 07:14:45.671674 17073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
Aug 10 07:14:45 dist-test-slave-10pc krb5kdc[16809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346085, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:14:45.679385 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 17073
I20260810 07:14:45.679492 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance
I20260810 07:14:45.679687 16229 external_mini_cluster.cc:1428] Setting key 7b731ee99cc067eb1c91cdd4de23ac3a
I20260810 07:14:45.681982 16900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:45.672486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:44791 (local address 127.15.217.126:38383)
0810 07:14:45.672561 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:45.672563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:45.672791 (+ 228us) server_negotiation.cc:407] Connection header received
0810 07:14:45.673835 (+ 1044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:45.673836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:45.673850 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:45.673904 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:45.674577 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:45.675098 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:45.676158 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:45.676357 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:45.676435 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:45.678722 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:45.678724 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:45.678724 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:45.678739 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:45.680420 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:45.680995 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:45.680996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:45.680996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:45.681049 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:45.681356 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:45.681357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:45.681357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:45.681460 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:45.681509 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:45.681850 (+ 341us) server_negotiation.cc:299] Negotiation successful
0810 07:14:45.681918 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 07:14:45.682334 17221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:45.672428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:38383 (local address 127.15.217.66:44791)
0810 07:14:45.672658 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:45.672702 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:45.673712 (+ 1010us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:45.673980 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:45.673984 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:45.674274 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:14:45.674476 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:45.674480 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:45.675203 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:45.675204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:45.676042 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:45.676044 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:45.676444 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:45.676445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:45.676611 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:45.677223 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:14:45.677232 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:14:45.678610 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:14:45.680542 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:45.680551 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:45.680554 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:45.680918 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:45.681118 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:45.681119 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:45.681119 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:45.681291 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:45.681525 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:14:45.681532 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:14:45.681710 (+ 178us) client_negotiation.cc:769] Sending connection context
0810 07:14:45.681832 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 07:14:45.682024 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":131,"threads_started":1}
I20260810 07:14:45.683068 17219 heartbeater.cc:347] Connected to a master server at 127.15.217.126:38383
I20260810 07:14:45.683188 17219 heartbeater.cc:464] Registering TS with master...
I20260810 07:14:45.683418 17219 heartbeater.cc:511] Master 127.15.217.126:38383 requested a full tablet report, sending...
I20260810 07:14:45.683862 16859 ts_manager.cc:194] Registered new tserver with Master: 9e83fdfc9f3a4f0e8e4e522f561fea24 (127.15.217.66:43607)
I20260810 07:14:45.684260 16859 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.66@KRBTEST.COM'} at 127.15.217.66:44791
WARNING: no policy specified for kudu/127.15.217.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab.
Entry for principal kudu/127.15.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab.
I20260810 07:14:45.707731 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.67:0
--local_ip_for_outbound_sockets=127.15.217.67
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:38383
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:45145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:14:45.795171 17224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:14:45.795434 17224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:14:45.795481 17224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:14:45.796810 17224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:14:45.796873 17224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:14:45.796919 17224 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.67
I20260810 07:14:45.798537 17224 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:45145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:38383
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.67
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:14:45.798846 17224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:14:45.799104 17224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:14:45.804282 17233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:45.804260 17230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:45.804260 17229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:14:45.804680 17224 server_base.cc:1034] running on GCE node
I20260810 07:14:45.804970 17224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:14:45.805230 17224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:14:45.806403 17224 hybrid_clock.cc:648] HybridClock initialized: now 1786346085806396 us; error 37 us; skew 500 ppm
Aug 10 07:14:45 dist-test-slave-10pc krb5kdc[16809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346085, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:14:45.809139 17224 init.cc:377] Logged in from keytab as kudu/127.15.217.67@KRBTEST.COM (short username kudu)
I20260810 07:14:45.809595 17224 webserver.cc:460] Webserver started at http://127.15.217.67:33673/ using document root <none> and password file <none>
I20260810 07:14:45.809852 17224 fs_manager.cc:359] Metadata directory not provided
I20260810 07:14:45.809924 17224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:14:45.810076 17224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:14:45.811173 17224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-2/data/instance:
uuid: "1fd273d65eee45a7b1ae64486b4289f8"
format_stamp: "Formatted at 2026-08-10 07:14:45 on dist-test-slave-10pc"
server_key: "d9788170fad27b8e706845bd1bfe045b"
server_key_iv: "5f2808d4dd95fc7ae93c1681f3a69b82"
server_key_version: "encryptionkey@0"
I20260810 07:14:45.811566 17224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance:
uuid: "1fd273d65eee45a7b1ae64486b4289f8"
format_stamp: "Formatted at 2026-08-10 07:14:45 on dist-test-slave-10pc"
server_key: "d9788170fad27b8e706845bd1bfe045b"
server_key_iv: "5f2808d4dd95fc7ae93c1681f3a69b82"
server_key_version: "encryptionkey@0"
I20260810 07:14:45.813053 17224 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:14:45.813849 17241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:45.814023 17224 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 07:14:45.814136 17224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-2/wal
uuid: "1fd273d65eee45a7b1ae64486b4289f8"
format_stamp: "Formatted at 2026-08-10 07:14:45 on dist-test-slave-10pc"
server_key: "d9788170fad27b8e706845bd1bfe045b"
server_key_iv: "5f2808d4dd95fc7ae93c1681f3a69b82"
server_key_version: "encryptionkey@0"
I20260810 07:14:45.814216 17224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:14:45.831600 17224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:14:45.833812 17224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:14:45.833985 17224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:14:45.834234 17224 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:14:45.834573 17224 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:14:45.834630 17224 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:45.834695 17224 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:14:45.834750 17224 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:45.841815 17224 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.67:44725
I20260810 07:14:45.841869 17371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.67:44725 every 8 connection(s)
I20260810 07:14:45.842231 17224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
I20260810 07:14:45.843894 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 17224
I20260810 07:14:45.843995 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance
I20260810 07:14:45.844152 16229 external_mini_cluster.cc:1428] Setting key f352ab5ad0f851a45a426f9731d42e71
Aug 10 07:14:45 dist-test-slave-10pc krb5kdc[16809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346085, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:14:45.852176 16900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:45.843058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:50665 (local address 127.15.217.126:38383)
0810 07:14:45.843126 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:45.843128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:45.843391 (+ 263us) server_negotiation.cc:407] Connection header received
0810 07:14:45.844575 (+ 1184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:45.844576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:45.844595 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:45.844654 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:45.845471 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:45.845980 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:45.847050 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:45.847230 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:45.847280 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:45.849699 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:45.849701 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:45.849701 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:45.849708 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:45.850916 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:45.851297 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:45.851297 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:45.851298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:45.851348 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:45.851561 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:45.851561 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:45.851561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:45.851670 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:45.851722 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:45.851958 (+ 236us) server_negotiation.cc:299] Negotiation successful
0810 07:14:45.852021 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 07:14:45.852396 17374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:45.843042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:38383 (local address 127.15.217.67:50665)
0810 07:14:45.843259 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:45.843299 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:45.844429 (+ 1130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:45.844737 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:45.844741 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:45.845150 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:14:45.845389 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:45.845394 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:45.846062 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:45.846062 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:45.846972 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:45.846974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:45.847308 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:45.847309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:45.847452 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:45.848003 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:14:45.848012 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:14:45.849613 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:14:45.850989 (+ 1376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:45.850996 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:45.850998 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:45.851232 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:45.851406 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:45.851407 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:45.851407 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:45.851508 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:45.851727 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:14:45.851732 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:14:45.851846 (+ 114us) client_negotiation.cc:769] Sending connection context
0810 07:14:45.851927 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 07:14:45.852091 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":127,"threads_started":1}
I20260810 07:14:45.852926 17372 heartbeater.cc:347] Connected to a master server at 127.15.217.126:38383
I20260810 07:14:45.853039 17372 heartbeater.cc:464] Registering TS with master...
I20260810 07:14:45.853248 17372 heartbeater.cc:511] Master 127.15.217.126:38383 requested a full tablet report, sending...
I20260810 07:14:45.853648 16859 ts_manager.cc:194] Registered new tserver with Master: 1fd273d65eee45a7b1ae64486b4289f8 (127.15.217.67:44725)
I20260810 07:14:45.854022 16859 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.67@KRBTEST.COM'} at 127.15.217.67:50665
I20260810 07:14:45.855852 16229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:14:45.863567 16900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:45.857452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60278 (local address 127.15.217.126:38383)
0810 07:14:45.857548 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:45.857551 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:45.857670 (+ 119us) server_negotiation.cc:407] Connection header received
0810 07:14:45.857748 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:45.857749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:45.857759 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:45.857814 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:45.858406 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:45.858886 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:45.859613 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:45.859812 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:45.859886 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:45.861157 (+ 1271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:45.861158 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:45.861159 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:45.861165 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:45.862327 (+ 1162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:45.862679 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:45.862680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:45.862681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:45.862744 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:45.862947 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:45.862948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:45.862949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:45.863113 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:45.863217 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:45.863364 (+ 147us) server_negotiation.cc:299] Negotiation successful
0810 07:14:45.863460 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 07:14:45.865926 16859 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60278:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:14:45.867591 16859 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:14:45.874493 17385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:45.870152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:43607 (local address 127.0.0.1:49618)
0810 07:14:45.870350 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:45.870392 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:45.870455 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:45.870850 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:45.870853 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:45.870856 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:14:45.870938 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:45.870940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:45.872000 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:45.872001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:45.873164 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:45.873167 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:45.874134 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:45.874135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:45.874220 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:45.874236 (+ 16us) client_negotiation.cc:769] Sending connection context
0810 07:14:45.874337 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 07:14:45.874404 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":98,"threads_started":1}
I20260810 07:14:45.874498 17387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:45.870411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49618 (local address 127.15.217.66:43607)
0810 07:14:45.870668 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:45.870674 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:45.870685 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:14:45.870727 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:45.870729 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:45.870746 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:45.870843 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:14:45.871026 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:45.871888 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:45.873264 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:45.874009 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:45.874111 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:45.874186 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:45.874365 (+ 179us) server_negotiation.cc:299] Negotiation successful
0810 07:14:45.874412 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":135,"threads_started":1}
I20260810 07:14:45.874821 17389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:45.870456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56712 (local address 127.15.217.67:44725)
0810 07:14:45.870668 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:45.870674 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:45.870981 (+ 307us) server_negotiation.cc:407] Connection header received
0810 07:14:45.871076 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:45.871079 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:45.871099 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:45.871198 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:14:45.871420 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:45.872255 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:45.873626 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:45.874309 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:45.874373 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:45.874430 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:45.874629 (+ 199us) server_negotiation.cc:299] Negotiation successful
0810 07:14:45.874684 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":112,"threads_started":1}
I20260810 07:14:45.875066 17386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:45.870152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:44725 (local address 127.0.0.1:56712)
0810 07:14:45.870833 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:45.870846 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:45.870916 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:45.871258 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:45.871260 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:45.871261 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:14:45.871302 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:45.871304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:45.872440 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:45.872441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:45.873533 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:45.873536 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:45.874466 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:45.874467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:45.874528 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:45.874532 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:14:45.874921 (+ 389us) client_negotiation.cc:240] Negotiation successful
0810 07:14:45.874983 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"mutex_wait_us":3,"spinlock_wait_cycles":384,"thread_start_us":101,"threads_started":1}
I20260810 07:14:45.875581 17146 tablet_service.cc:1467] Processing CreateTablet for tablet 2cb91870c1a9410ebc1f9616041516f0 (DEFAULT_TABLE table=test-table [id=e9cc498b7f8345e283c9a63f2a06da66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:14:45.875780 17296 tablet_service.cc:1467] Processing CreateTablet for tablet 2cb91870c1a9410ebc1f9616041516f0 (DEFAULT_TABLE table=test-table [id=e9cc498b7f8345e283c9a63f2a06da66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:14:45.875979 17146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cb91870c1a9410ebc1f9616041516f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:14:45.876030 17296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cb91870c1a9410ebc1f9616041516f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:14:45.876716 17384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:45.870158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:41959 (local address 127.0.0.1:35272)
0810 07:14:45.872497 (+ 2339us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:45.872509 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:45.872563 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:45.872755 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:45.872756 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:45.872758 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:14:45.872846 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:45.872847 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:45.874171 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:45.874172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:45.875485 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:45.875488 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:45.876447 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:45.876467 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:45.876528 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:45.876534 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 07:14:45.876605 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 07:14:45.876645 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2323,"thread_start_us":69,"threads_started":1}
I20260810 07:14:45.876781 17391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:45.870776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35272 (local address 127.15.217.65:41959)
0810 07:14:45.871030 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:45.871035 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:45.872572 (+ 1537us) server_negotiation.cc:407] Connection header received
0810 07:14:45.872630 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:45.872632 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:45.872649 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:45.872735 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:14:45.872932 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:45.873990 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:45.875578 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:45.876335 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:45.876414 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:45.876516 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:45.876620 (+ 104us) server_negotiation.cc:299] Negotiation successful
0810 07:14:45.876662 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":120,"threads_started":1}
I20260810 07:14:45.879334 17392 tablet_bootstrap.cc:492] T 2cb91870c1a9410ebc1f9616041516f0 P 9e83fdfc9f3a4f0e8e4e522f561fea24: Bootstrap starting.
I20260810 07:14:45.879789 17393 tablet_bootstrap.cc:492] T 2cb91870c1a9410ebc1f9616041516f0 P 1fd273d65eee45a7b1ae64486b4289f8: Bootstrap starting.
I20260810 07:14:45.880889 17392 tablet_bootstrap.cc:654] T 2cb91870c1a9410ebc1f9616041516f0 P 9e83fdfc9f3a4f0e8e4e522f561fea24: Neither blocks nor log segments found. Creating new log.
I20260810 07:14:45.881024 16996 tablet_service.cc:1467] Processing CreateTablet for tablet 2cb91870c1a9410ebc1f9616041516f0 (DEFAULT_TABLE table=test-table [id=e9cc498b7f8345e283c9a63f2a06da66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:14:45.881361 17393 tablet_bootstrap.cc:654] T 2cb91870c1a9410ebc1f9616041516f0 P 1fd273d65eee45a7b1ae64486b4289f8: Neither blocks nor log segments found. Creating new log.
I20260810 07:14:45.881450 16996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cb91870c1a9410ebc1f9616041516f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:14:45.882043 17392 log.cc:824] T 2cb91870c1a9410ebc1f9616041516f0 P 9e83fdfc9f3a4f0e8e4e522f561fea24: Log is configured to *not* fsync() on all Append() calls
I20260810 07:14:45.882676 17393 log.cc:824] T 2cb91870c1a9410ebc1f9616041516f0 P 1fd273d65eee45a7b1ae64486b4289f8: Log is configured to *not* fsync() on all Append() calls
I20260810 07:14:45.883061 17392 tablet_bootstrap.cc:492] T 2cb91870c1a9410ebc1f9616041516f0 P 9e83fdfc9f3a4f0e8e4e522f561fea24: No bootstrap required, opened a new log
I20260810 07:14:45.883236 17392 ts_tablet_manager.cc:1397] T 2cb91870c1a9410ebc1f9616041516f0 P 9e83fdfc9f3a4f0e8e4e522f561fea24: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.004s
I20260810 07:14:45.883831 17393 tablet_bootstrap.cc:492] T 2cb91870c1a9410ebc1f9616041516f0 P 1fd273d65eee45a7b1ae64486b4289f8: No bootstrap required, opened a new log
I20260810 07:14:45.883961 17393 ts_tablet_manager.cc:1397] T 2cb91870c1a9410ebc1f9616041516f0 P 1fd273d65eee45a7b1ae64486b4289f8: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260810 07:14:45.886487 17396 tablet_bootstrap.cc:492] T 2cb91870c1a9410ebc1f9616041516f0 P 529b70e09ebb412d8b2f0fa4303eb4f8: Bootstrap starting.
I20260810 07:14:45.887784 17396 tablet_bootstrap.cc:654] T 2cb91870c1a9410ebc1f9616041516f0 P 529b70e09ebb412d8b2f0fa4303eb4f8: Neither blocks nor log segments found. Creating new log.
I20260810 07:14:45.887492 17392 raft_consensus.cc:348] T 2cb91870c1a9410ebc1f9616041516f0 P 9e83fdfc9f3a4f0e8e4e522f561fea24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e83fdfc9f3a4f0e8e4e522f561fea24" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 43607 } } peers { permanent_uuid: "1fd273d65eee45a7b1ae64486b4289f8" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 44725 } } peers { permanent_uuid: "529b70e09ebb412d8b2f0fa4303eb4f8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41959 } }
I20260810 07:14:45.887876 17392 raft_consensus.cc:374] T 2cb91870c1a9410ebc1f9616041516f0 P 9e83fdfc9f3a4f0e8e4e522f561fea24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:14:45.887957 17392 raft_consensus.cc:729] T 2cb91870c1a9410ebc1f9616041516f0 P 9e83fdfc9f3a4f0e8e4e522f561fea24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e83fdfc9f3a4f0e8e4e522f561fea24, State: Initialized, Role: FOLLOWER
I20260810 07:14:45.888152 17392 consensus_queue.cc:260] T 2cb91870c1a9410ebc1f9616041516f0 P 9e83fdfc9f3a4f0e8e4e522f561fea24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e83fdfc9f3a4f0e8e4e522f561fea24" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 43607 } } peers { permanent_uuid: "1fd273d65eee45a7b1ae64486b4289f8" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 44725 } } peers { permanent_uuid: "529b70e09ebb412d8b2f0fa4303eb4f8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41959 } }
I20260810 07:14:45.888749 17392 ts_tablet_manager.cc:1428] T 2cb91870c1a9410ebc1f9616041516f0 P 9e83fdfc9f3a4f0e8e4e522f561fea24: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260810 07:14:45.888809 17219 heartbeater.cc:503] Master 127.15.217.126:38383 was elected leader, sending a full tablet report...
I20260810 07:14:45.889104 17396 log.cc:824] T 2cb91870c1a9410ebc1f9616041516f0 P 529b70e09ebb412d8b2f0fa4303eb4f8: Log is configured to *not* fsync() on all Append() calls
I20260810 07:14:45.888953 17393 raft_consensus.cc:348] T 2cb91870c1a9410ebc1f9616041516f0 P 1fd273d65eee45a7b1ae64486b4289f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e83fdfc9f3a4f0e8e4e522f561fea24" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 43607 } } peers { permanent_uuid: "1fd273d65eee45a7b1ae64486b4289f8" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 44725 } } peers { permanent_uuid: "529b70e09ebb412d8b2f0fa4303eb4f8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41959 } }
I20260810 07:14:45.889215 17393 raft_consensus.cc:374] T 2cb91870c1a9410ebc1f9616041516f0 P 1fd273d65eee45a7b1ae64486b4289f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:14:45.889261 17393 raft_consensus.cc:729] T 2cb91870c1a9410ebc1f9616041516f0 P 1fd273d65eee45a7b1ae64486b4289f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fd273d65eee45a7b1ae64486b4289f8, State: Initialized, Role: FOLLOWER
I20260810 07:14:45.889375 17393 consensus_queue.cc:260] T 2cb91870c1a9410ebc1f9616041516f0 P 1fd273d65eee45a7b1ae64486b4289f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e83fdfc9f3a4f0e8e4e522f561fea24" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 43607 } } peers { permanent_uuid: "1fd273d65eee45a7b1ae64486b4289f8" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 44725 } } peers { permanent_uuid: "529b70e09ebb412d8b2f0fa4303eb4f8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41959 } }
I20260810 07:14:45.889854 17393 ts_tablet_manager.cc:1428] T 2cb91870c1a9410ebc1f9616041516f0 P 1fd273d65eee45a7b1ae64486b4289f8: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260810 07:14:45.889850 17372 heartbeater.cc:503] Master 127.15.217.126:38383 was elected leader, sending a full tablet report...
I20260810 07:14:45.890113 17396 tablet_bootstrap.cc:492] T 2cb91870c1a9410ebc1f9616041516f0 P 529b70e09ebb412d8b2f0fa4303eb4f8: No bootstrap required, opened a new log
I20260810 07:14:45.890198 17396 ts_tablet_manager.cc:1397] T 2cb91870c1a9410ebc1f9616041516f0 P 529b70e09ebb412d8b2f0fa4303eb4f8: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260810 07:14:45.893575 17396 raft_consensus.cc:348] T 2cb91870c1a9410ebc1f9616041516f0 P 529b70e09ebb412d8b2f0fa4303eb4f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e83fdfc9f3a4f0e8e4e522f561fea24" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 43607 } } peers { permanent_uuid: "1fd273d65eee45a7b1ae64486b4289f8" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 44725 } } peers { permanent_uuid: "529b70e09ebb412d8b2f0fa4303eb4f8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41959 } }
I20260810 07:14:45.893764 17396 raft_consensus.cc:374] T 2cb91870c1a9410ebc1f9616041516f0 P 529b70e09ebb412d8b2f0fa4303eb4f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:14:45.893816 17396 raft_consensus.cc:729] T 2cb91870c1a9410ebc1f9616041516f0 P 529b70e09ebb412d8b2f0fa4303eb4f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 529b70e09ebb412d8b2f0fa4303eb4f8, State: Initialized, Role: FOLLOWER
I20260810 07:14:45.893937 17396 consensus_queue.cc:260] T 2cb91870c1a9410ebc1f9616041516f0 P 529b70e09ebb412d8b2f0fa4303eb4f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e83fdfc9f3a4f0e8e4e522f561fea24" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 43607 } } peers { permanent_uuid: "1fd273d65eee45a7b1ae64486b4289f8" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 44725 } } peers { permanent_uuid: "529b70e09ebb412d8b2f0fa4303eb4f8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41959 } }
I20260810 07:14:45.894367 17396 ts_tablet_manager.cc:1428] T 2cb91870c1a9410ebc1f9616041516f0 P 529b70e09ebb412d8b2f0fa4303eb4f8: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260810 07:14:45.894397 17068 heartbeater.cc:503] Master 127.15.217.126:38383 was elected leader, sending a full tablet report...
W20260810 07:14:45.922394 17220 tablet.cc:2369] T 2cb91870c1a9410ebc1f9616041516f0 P 9e83fdfc9f3a4f0e8e4e522f561fea24: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:14:45.982734 17397 raft_consensus.cc:482] T 2cb91870c1a9410ebc1f9616041516f0 P 9e83fdfc9f3a4f0e8e4e522f561fea24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:14:45.982954 17397 raft_consensus.cc:504] T 2cb91870c1a9410ebc1f9616041516f0 P 9e83fdfc9f3a4f0e8e4e522f561fea24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e83fdfc9f3a4f0e8e4e522f561fea24" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 43607 } } peers { permanent_uuid: "1fd273d65eee45a7b1ae64486b4289f8" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 44725 } } peers { permanent_uuid: "529b70e09ebb412d8b2f0fa4303eb4f8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41959 } }
I20260810 07:14:45.983477 17397 leader_election.cc:288] T 2cb91870c1a9410ebc1f9616041516f0 P 9e83fdfc9f3a4f0e8e4e522f561fea24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fd273d65eee45a7b1ae64486b4289f8 (127.15.217.67:44725), 529b70e09ebb412d8b2f0fa4303eb4f8 (127.15.217.65:41959)
I20260810 07:14:45.987509 17389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:45.983849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:36619 (local address 127.15.217.67:44725)
0810 07:14:45.983894 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:45.983896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:45.983911 (+ 15us) server_negotiation.cc:407] Connection header received
0810 07:14:45.984017 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:45.984019 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:45.984028 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:45.984082 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:14:45.984407 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:45.985104 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:45.986534 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:45.987071 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:45.987171 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:45.987183 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:45.987397 (+ 214us) server_negotiation.cc:299] Negotiation successful
0810 07:14:45.987440 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 07:14:45.987509 17221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:45.983777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:44725 (local address 127.15.217.66:36619)
0810 07:14:45.983842 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:45.983857 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:45.983932 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:45.984084 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:45.984086 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:45.984087 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:14:45.984291 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:45.984295 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:45.985190 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:45.985191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:45.986405 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:45.986408 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:45.987162 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:45.987163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:45.987309 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:45.987313 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:14:45.987387 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 07:14:45.987434 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 07:14:45.987848 17401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:45.983786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:41959 (local address 127.15.217.66:57247)
0810 07:14:45.984160 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:45.984175 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:45.984247 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:45.984412 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:45.984413 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:45.984415 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:14:45.984638 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:45.984640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:45.985396 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:45.985397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:45.986829 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:45.986832 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:45.987425 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:45.987426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:45.987557 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:45.987579 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 07:14:45.987674 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 07:14:45.987730 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":75,"thread_start_us":147,"threads_started":1}
I20260810 07:14:45.987823 17391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:45.983784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:57247 (local address 127.15.217.65:41959)
0810 07:14:45.983868 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:45.983871 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:45.984291 (+ 420us) server_negotiation.cc:407] Connection header received
0810 07:14:45.984326 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:45.984327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:45.984337 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:45.984391 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:14:45.984722 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:45.985305 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:45.986921 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:45.987323 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:45.987410 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:45.987419 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:45.987693 (+ 274us) server_negotiation.cc:299] Negotiation successful
0810 07:14:45.987752 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 07:14:45.988162 17319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb91870c1a9410ebc1f9616041516f0" candidate_uuid: "9e83fdfc9f3a4f0e8e4e522f561fea24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fd273d65eee45a7b1ae64486b4289f8" is_pre_election: true
I20260810 07:14:45.988407 17319 raft_consensus.cc:2454] T 2cb91870c1a9410ebc1f9616041516f0 P 1fd273d65eee45a7b1ae64486b4289f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e83fdfc9f3a4f0e8e4e522f561fea24 in term 0.
I20260810 07:14:45.988591 17016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb91870c1a9410ebc1f9616041516f0" candidate_uuid: "9e83fdfc9f3a4f0e8e4e522f561fea24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "529b70e09ebb412d8b2f0fa4303eb4f8" is_pre_election: true
I20260810 07:14:45.988883 17016 raft_consensus.cc:2454] T 2cb91870c1a9410ebc1f9616041516f0 P 529b70e09ebb412d8b2f0fa4303eb4f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e83fdfc9f3a4f0e8e4e522f561fea24 in term 0.
I20260810 07:14:45.988829 17095 leader_election.cc:302] T 2cb91870c1a9410ebc1f9616041516f0 P 9e83fdfc9f3a4f0e8e4e522f561fea24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fd273d65eee45a7b1ae64486b4289f8, 9e83fdfc9f3a4f0e8e4e522f561fea24; no voters:
I20260810 07:14:45.989039 17397 raft_consensus.cc:2781] T 2cb91870c1a9410ebc1f9616041516f0 P 9e83fdfc9f3a4f0e8e4e522f561fea24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:14:45.989107 17397 raft_consensus.cc:482] T 2cb91870c1a9410ebc1f9616041516f0 P 9e83fdfc9f3a4f0e8e4e522f561fea24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:14:45.989133 17397 raft_consensus.cc:3037] T 2cb91870c1a9410ebc1f9616041516f0 P 9e83fdfc9f3a4f0e8e4e522f561fea24 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:14:45.990123 17397 raft_consensus.cc:504] T 2cb91870c1a9410ebc1f9616041516f0 P 9e83fdfc9f3a4f0e8e4e522f561fea24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e83fdfc9f3a4f0e8e4e522f561fea24" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 43607 } } peers { permanent_uuid: "1fd273d65eee45a7b1ae64486b4289f8" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 44725 } } peers { permanent_uuid: "529b70e09ebb412d8b2f0fa4303eb4f8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41959 } }
I20260810 07:14:45.990288 17397 leader_election.cc:288] T 2cb91870c1a9410ebc1f9616041516f0 P 9e83fdfc9f3a4f0e8e4e522f561fea24 [CANDIDATE]: Term 1 election: Requested vote from peers 1fd273d65eee45a7b1ae64486b4289f8 (127.15.217.67:44725), 529b70e09ebb412d8b2f0fa4303eb4f8 (127.15.217.65:41959)
I20260810 07:14:45.990440 17016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb91870c1a9410ebc1f9616041516f0" candidate_uuid: "9e83fdfc9f3a4f0e8e4e522f561fea24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "529b70e09ebb412d8b2f0fa4303eb4f8"
I20260810 07:14:45.990463 17319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb91870c1a9410ebc1f9616041516f0" candidate_uuid: "9e83fdfc9f3a4f0e8e4e522f561fea24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fd273d65eee45a7b1ae64486b4289f8"
I20260810 07:14:45.990556 17016 raft_consensus.cc:3037] T 2cb91870c1a9410ebc1f9616041516f0 P 529b70e09ebb412d8b2f0fa4303eb4f8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:14:45.990551 17319 raft_consensus.cc:3037] T 2cb91870c1a9410ebc1f9616041516f0 P 1fd273d65eee45a7b1ae64486b4289f8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:14:45.991385 17016 raft_consensus.cc:2454] T 2cb91870c1a9410ebc1f9616041516f0 P 529b70e09ebb412d8b2f0fa4303eb4f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e83fdfc9f3a4f0e8e4e522f561fea24 in term 1.
I20260810 07:14:45.991470 17319 raft_consensus.cc:2454] T 2cb91870c1a9410ebc1f9616041516f0 P 1fd273d65eee45a7b1ae64486b4289f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e83fdfc9f3a4f0e8e4e522f561fea24 in term 1.
I20260810 07:14:45.991654 17091 leader_election.cc:302] T 2cb91870c1a9410ebc1f9616041516f0 P 9e83fdfc9f3a4f0e8e4e522f561fea24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 529b70e09ebb412d8b2f0fa4303eb4f8, 9e83fdfc9f3a4f0e8e4e522f561fea24; no voters:
I20260810 07:14:45.991770 17397 raft_consensus.cc:2781] T 2cb91870c1a9410ebc1f9616041516f0 P 9e83fdfc9f3a4f0e8e4e522f561fea24 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:14:45.991933 17397 raft_consensus.cc:686] T 2cb91870c1a9410ebc1f9616041516f0 P 9e83fdfc9f3a4f0e8e4e522f561fea24 [term 1 LEADER]: Becoming Leader. State: Replica: 9e83fdfc9f3a4f0e8e4e522f561fea24, State: Running, Role: LEADER
I20260810 07:14:45.992038 17397 consensus_queue.cc:237] T 2cb91870c1a9410ebc1f9616041516f0 P 9e83fdfc9f3a4f0e8e4e522f561fea24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e83fdfc9f3a4f0e8e4e522f561fea24" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 43607 } } peers { permanent_uuid: "1fd273d65eee45a7b1ae64486b4289f8" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 44725 } } peers { permanent_uuid: "529b70e09ebb412d8b2f0fa4303eb4f8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41959 } }
I20260810 07:14:45.993054 16858 catalog_manager.cc:5495] T 2cb91870c1a9410ebc1f9616041516f0 P 9e83fdfc9f3a4f0e8e4e522f561fea24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e83fdfc9f3a4f0e8e4e522f561fea24 (127.15.217.66). New cstate: current_term: 1 leader_uuid: "9e83fdfc9f3a4f0e8e4e522f561fea24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e83fdfc9f3a4f0e8e4e522f561fea24" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 43607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fd273d65eee45a7b1ae64486b4289f8" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 44725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "529b70e09ebb412d8b2f0fa4303eb4f8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41959 } health_report { overall_health: UNKNOWN } } }
W20260810 07:14:46.010771 17069 tablet.cc:2369] T 2cb91870c1a9410ebc1f9616041516f0 P 529b70e09ebb412d8b2f0fa4303eb4f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 07:14:46 dist-test-slave-10pc krb5kdc[16809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.65@KRBTEST.COM
I20260810 07:14:46.024624 17391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:46.016147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35282 (local address 127.15.217.65:41959)
0810 07:14:46.016238 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:46.016244 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:46.016362 (+ 118us) server_negotiation.cc:407] Connection header received
0810 07:14:46.016415 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:46.016416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:46.016426 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:46.016510 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:46.017276 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:46.017756 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:46.018609 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:46.018793 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:46.018846 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:46.021466 (+ 2620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:46.021478 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:46.021479 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:46.021486 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:46.023391 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:46.023734 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:46.023738 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:46.023738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:46.023797 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:46.024016 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:46.024016 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:46.024017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:46.024189 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:46.024310 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:46.024372 (+ 62us) server_negotiation.cc:299] Negotiation successful
0810 07:14:46.024440 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Aug 10 07:14:46 dist-test-slave-10pc krb5kdc[16809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346086, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 07:14:46 dist-test-slave-10pc krb5kdc[16809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346086, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.217.65@KRBTEST.COM
I20260810 07:14:46.051676 17391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:46.043782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35286 (local address 127.15.217.65:41959)
0810 07:14:46.043870 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:46.043873 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:46.043955 (+ 82us) server_negotiation.cc:407] Connection header received
0810 07:14:46.044065 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:46.044066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:46.044083 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:46.044157 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:46.044869 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:46.045453 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:46.046213 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:46.046416 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:46.046497 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:46.049050 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:46.049052 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:46.049052 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:46.049060 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:46.050366 (+ 1306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:46.050733 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:46.050735 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:46.050735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:46.050805 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:46.051019 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:46.051020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:46.051020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:46.051134 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:46.051248 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:46.051475 (+ 227us) server_negotiation.cc:299] Negotiation successful
0810 07:14:46.051566 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Aug 10 07:14:46 dist-test-slave-10pc krb5kdc[16809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346086, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 07:14:46 dist-test-slave-10pc krb5kdc[16809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346086, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.65@KRBTEST.COM
I20260810 07:14:46.077373 17391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:46.069947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35288 (local address 127.15.217.65:41959)
0810 07:14:46.070025 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:46.070028 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:46.070142 (+ 114us) server_negotiation.cc:407] Connection header received
0810 07:14:46.070237 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:46.070238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:46.070252 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:46.070316 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:46.070964 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:46.071544 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:46.072340 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:46.072609 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:46.072689 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:46.074927 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:46.074928 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:46.074929 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:46.074937 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:46.076167 (+ 1230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:46.076561 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:46.076562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:46.076563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:46.076627 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:46.076839 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:46.076840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:46.076841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:46.076949 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:46.077060 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:46.077179 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 07:14:46.077262 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260810 07:14:46.077889 16972 tablet_service.cc:2908] Rejecting scan request for tablet 2cb91870c1a9410ebc1f9616041516f0: Uninitialized: clean time has not yet been initialized
I20260810 07:14:46.078831 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 16925
I20260810 07:14:46.087679 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 17073
W20260810 07:14:46.092999 17373 tablet.cc:2369] T 2cb91870c1a9410ebc1f9616041516f0 P 1fd273d65eee45a7b1ae64486b4289f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:14:46.097699 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 17224
I20260810 07:14:46.105329 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 16824
2026-08-10T07:14:46Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2198 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:14:46 dist-test-slave-10pc krb5kdc[17418](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:14:46 dist-test-slave-10pc krb5kdc[17418](info): set up 2 sockets
Aug 10 07:14:46 dist-test-slave-10pc krb5kdc[17418](info): commencing operation
krb5kdc: starting...
W20260810 07:14:47.215716 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:14:47 dist-test-slave-10pc krb5kdc[17418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:14:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:14:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
I20260810 07:14:47.347601 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:35039
--webserver_interface=127.15.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:40439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:35039
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:14:47.434331 17433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:14:47.434593 17433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:14:47.434654 17433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:14:47.435889 17433 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:14:47.435971 17433 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 07:14:47.436000 17433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:14:47.436022 17433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:14:47.436056 17433 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:14:47.436080 17433 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:14:47.437662 17433 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:40439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:35039
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:35039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:14:47.437968 17433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:14:47.438222 17433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:14:47.443215 17442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:47.443291 17439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:47.443233 17438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:14:47.443483 17433 server_base.cc:1034] running on GCE node
I20260810 07:14:47.443761 17433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:14:47.444061 17433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:14:47.445247 17433 hybrid_clock.cc:648] HybridClock initialized: now 1786346087445248 us; error 67 us; skew 500 ppm
Aug 10 07:14:47 dist-test-slave-10pc krb5kdc[17418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346087, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:14:47.452525 17433 init.cc:377] Logged in from keytab as kudu/127.15.217.126@KRBTEST.COM (short username kudu)
I20260810 07:14:47.453007 17433 webserver.cc:460] Webserver started at http://127.15.217.126:41909/ using document root <none> and password file <none>
I20260810 07:14:47.453295 17433 fs_manager.cc:359] Metadata directory not provided
I20260810 07:14:47.453369 17433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:14:47.453524 17433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:14:47.454644 17433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "36a23f9d47b9445884702e5af00539f6"
format_stamp: "Formatted at 2026-08-10 07:14:47 on dist-test-slave-10pc"
server_key: "d74ff256689daf11b4f24a3b909c262a"
server_key_iv: "5d996edae69be3abfc563063e9bdd6d3"
server_key_version: "encryptionkey@0"
I20260810 07:14:47.455039 17433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "36a23f9d47b9445884702e5af00539f6"
format_stamp: "Formatted at 2026-08-10 07:14:47 on dist-test-slave-10pc"
server_key: "d74ff256689daf11b4f24a3b909c262a"
server_key_iv: "5d996edae69be3abfc563063e9bdd6d3"
server_key_version: "encryptionkey@0"
I20260810 07:14:47.456431 17433 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:14:47.457316 17450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:47.457628 17433 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:14:47.457733 17433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "36a23f9d47b9445884702e5af00539f6"
format_stamp: "Formatted at 2026-08-10 07:14:47 on dist-test-slave-10pc"
server_key: "d74ff256689daf11b4f24a3b909c262a"
server_key_iv: "5d996edae69be3abfc563063e9bdd6d3"
server_key_version: "encryptionkey@0"
I20260810 07:14:47.457813 17433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:14:47.478217 17433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:14:47.480378 17433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:14:47.480589 17433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:14:47.485790 17433 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:35039
I20260810 07:14:47.485886 17520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:35039 every 8 connection(s)
I20260810 07:14:47.486279 17433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:14:47.487139 17521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:14:47.489912 17521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36a23f9d47b9445884702e5af00539f6: Bootstrap starting.
I20260810 07:14:47.490635 17521 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36a23f9d47b9445884702e5af00539f6: Neither blocks nor log segments found. Creating new log.
I20260810 07:14:47.490985 17521 log.cc:824] T 00000000000000000000000000000000 P 36a23f9d47b9445884702e5af00539f6: Log is configured to *not* fsync() on all Append() calls
I20260810 07:14:47.491822 17521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36a23f9d47b9445884702e5af00539f6: No bootstrap required, opened a new log
I20260810 07:14:47.494148 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 17433
I20260810 07:14:47.494230 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:14:47.494421 16229 external_mini_cluster.cc:1428] Setting key fd65d87c42b7853b9ed86011bab60c00
I20260810 07:14:47.494642 17521 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36a23f9d47b9445884702e5af00539f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a23f9d47b9445884702e5af00539f6" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35039 } }
I20260810 07:14:47.494869 17521 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36a23f9d47b9445884702e5af00539f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:14:47.495000 17521 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36a23f9d47b9445884702e5af00539f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36a23f9d47b9445884702e5af00539f6, State: Initialized, Role: FOLLOWER
I20260810 07:14:47.495170 17521 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36a23f9d47b9445884702e5af00539f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a23f9d47b9445884702e5af00539f6" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35039 } }
I20260810 07:14:47.495297 17521 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36a23f9d47b9445884702e5af00539f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:14:47.495339 17521 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36a23f9d47b9445884702e5af00539f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:14:47.495397 17521 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36a23f9d47b9445884702e5af00539f6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:14:47.496198 17521 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36a23f9d47b9445884702e5af00539f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a23f9d47b9445884702e5af00539f6" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35039 } }
I20260810 07:14:47.496302 17521 leader_election.cc:302] T 00000000000000000000000000000000 P 36a23f9d47b9445884702e5af00539f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36a23f9d47b9445884702e5af00539f6; no voters:
I20260810 07:14:47.496526 17521 leader_election.cc:288] T 00000000000000000000000000000000 P 36a23f9d47b9445884702e5af00539f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:14:47.496556 17526 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36a23f9d47b9445884702e5af00539f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:14:47.496681 17526 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36a23f9d47b9445884702e5af00539f6 [term 1 LEADER]: Becoming Leader. State: Replica: 36a23f9d47b9445884702e5af00539f6, State: Running, Role: LEADER
I20260810 07:14:47.496836 17521 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36a23f9d47b9445884702e5af00539f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:14:47.496845 17526 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36a23f9d47b9445884702e5af00539f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a23f9d47b9445884702e5af00539f6" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35039 } }
I20260810 07:14:47.497499 17527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36a23f9d47b9445884702e5af00539f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36a23f9d47b9445884702e5af00539f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a23f9d47b9445884702e5af00539f6" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35039 } } }
I20260810 07:14:47.497617 17527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36a23f9d47b9445884702e5af00539f6 [sys.catalog]: This master's current role is: LEADER
I20260810 07:14:47.497514 17529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36a23f9d47b9445884702e5af00539f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36a23f9d47b9445884702e5af00539f6. Latest consensus state: current_term: 1 leader_uuid: "36a23f9d47b9445884702e5af00539f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36a23f9d47b9445884702e5af00539f6" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35039 } } }
I20260810 07:14:47.497854 17529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36a23f9d47b9445884702e5af00539f6 [sys.catalog]: This master's current role is: LEADER
I20260810 07:14:47.498292 17534 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:14:47.498950 17534 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:14:47.500865 17534 catalog_manager.cc:1347] Generated new cluster ID: db78fe4633774c089a21ff90fd3c2666
I20260810 07:14:47.500926 17534 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 07:14:47 dist-test-slave-10pc krb5kdc[17418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:14:47.507306 17525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:47.495138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52118 (local address 127.15.217.126:35039)
0810 07:14:47.495371 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:47.495382 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:47.495421 (+ 39us) server_negotiation.cc:407] Connection header received
0810 07:14:47.495585 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:47.495593 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:47.495656 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:47.495818 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:47.496526 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:47.497855 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:47.499124 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:47.499852 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:47.500014 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:47.503170 (+ 3156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:47.503174 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:47.503175 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:47.503188 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:47.505382 (+ 2194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:47.505864 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:47.505868 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:47.505868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:47.506021 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:47.506260 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:47.506261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:47.506261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:47.506599 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:47.506783 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:47.506901 (+ 118us) server_negotiation.cc:299] Negotiation successful
0810 07:14:47.507061 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":107,"threads_started":1}
I20260810 07:14:47.513700 17534 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:14:47.514307 17534 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:14:47.518244 17534 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36a23f9d47b9445884702e5af00539f6: Generated new TSK 0
I20260810 07:14:47.518447 17534 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab.
Entry for principal kudu/127.15.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab.
I20260810 07:14:47.556077 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.65:0
--local_ip_for_outbound_sockets=127.15.217.65
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:35039
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:40439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:14:47.643616 17550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:14:47.643837 17550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:14:47.643862 17550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:14:47.645025 17550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:14:47.645066 17550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:14:47.645085 17550 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 07:14:47.645103 17550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.65
I20260810 07:14:47.646559 17550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:40439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.217.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.217.126:35039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.65
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:14:47.646765 17550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:14:47.646967 17550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:14:47.652232 17555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:47.652232 17558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:47.652240 17556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:14:47.652540 17550 server_base.cc:1034] running on GCE node
I20260810 07:14:47.652767 17550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:14:47.652968 17550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:14:47.654139 17550 hybrid_clock.cc:648] HybridClock initialized: now 1786346087654127 us; error 38 us; skew 500 ppm
Aug 10 07:14:47 dist-test-slave-10pc krb5kdc[17418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346087, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:14:47.656741 17550 init.cc:377] Logged in from keytab as kudu/127.15.217.65@KRBTEST.COM (short username kudu)
I20260810 07:14:47.657235 17550 webserver.cc:460] Webserver started at http://127.15.217.65:35811/ using document root <none> and password file <none>
I20260810 07:14:47.657495 17550 fs_manager.cc:359] Metadata directory not provided
I20260810 07:14:47.657567 17550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:14:47.657727 17550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:14:47.658823 17550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-0/data/instance:
uuid: "29fc149bdd0f49db9423c9ba867133ce"
format_stamp: "Formatted at 2026-08-10 07:14:47 on dist-test-slave-10pc"
server_key: "0d79c05c87799c1fed4551754fad7115"
server_key_iv: "3f861bf88889ff4a7cb77d14a9ad5f3a"
server_key_version: "encryptionkey@0"
I20260810 07:14:47.659215 17550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance:
uuid: "29fc149bdd0f49db9423c9ba867133ce"
format_stamp: "Formatted at 2026-08-10 07:14:47 on dist-test-slave-10pc"
server_key: "0d79c05c87799c1fed4551754fad7115"
server_key_iv: "3f861bf88889ff4a7cb77d14a9ad5f3a"
server_key_version: "encryptionkey@0"
I20260810 07:14:47.660679 17550 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:14:47.661520 17566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:47.661721 17550 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:14:47.661839 17550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-0/wal
uuid: "29fc149bdd0f49db9423c9ba867133ce"
format_stamp: "Formatted at 2026-08-10 07:14:47 on dist-test-slave-10pc"
server_key: "0d79c05c87799c1fed4551754fad7115"
server_key_iv: "3f861bf88889ff4a7cb77d14a9ad5f3a"
server_key_version: "encryptionkey@0"
I20260810 07:14:47.661935 17550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:14:47.669912 17550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:14:47.671777 17550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:14:47.671947 17550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:14:47.672541 17550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:14:47.672605 17550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:47.672648 17550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:14:47.672708 17550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 07:14:47 dist-test-slave-10pc krb5kdc[17418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346087, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:14:47.681309 17550 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.65:34917
I20260810 07:14:47.681414 17699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.65:34917 every 8 connection(s)
I20260810 07:14:47.681874 17550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
I20260810 07:14:47.682401 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 17550
I20260810 07:14:47.682509 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance
I20260810 07:14:47.682696 16229 external_mini_cluster.cc:1428] Setting key 2753ea76ad53b635c76f7b5f65875b3f
I20260810 07:14:47.683674 17525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:47.673140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:35889 (local address 127.15.217.126:35039)
0810 07:14:47.673219 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:47.673223 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:47.673419 (+ 196us) server_negotiation.cc:407] Connection header received
0810 07:14:47.673917 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:47.673918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:47.673933 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:47.674022 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:47.674847 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:47.675365 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:47.676384 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:47.676671 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:47.676751 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:47.680102 (+ 3351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:47.680103 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:47.680104 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:47.680112 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:47.681890 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:47.682445 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:47.682447 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:47.682447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:47.682520 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:47.682829 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:47.682830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:47.682831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:47.683020 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:47.683090 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:47.683434 (+ 344us) server_negotiation.cc:299] Negotiation successful
0810 07:14:47.683547 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 07:14:47.683758 17576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:47.673103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35039 (local address 127.15.217.65:35889)
0810 07:14:47.673287 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:47.673328 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:47.673820 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:47.674172 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:47.674177 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:47.674524 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:14:47.674730 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:47.674735 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:47.675473 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:47.675474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:47.676290 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:47.676292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:47.676762 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:47.676763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:47.676920 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:47.677520 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:14:47.677532 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:14:47.679988 (+ 2456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:14:47.682001 (+ 2013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:47.682008 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:47.682010 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:47.682357 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:47.682596 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:47.682597 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:47.682597 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:47.682746 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:47.683082 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:14:47.683086 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:14:47.683244 (+ 158us) client_negotiation.cc:769] Sending connection context
0810 07:14:47.683391 (+ 147us) client_negotiation.cc:240] Negotiation successful
0810 07:14:47.683561 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":109,"threads_started":1}
I20260810 07:14:47.684356 17700 heartbeater.cc:347] Connected to a master server at 127.15.217.126:35039
I20260810 07:14:47.684491 17700 heartbeater.cc:464] Registering TS with master...
I20260810 07:14:47.684690 17700 heartbeater.cc:511] Master 127.15.217.126:35039 requested a full tablet report, sending...
I20260810 07:14:47.685338 17470 ts_manager.cc:194] Registered new tserver with Master: 29fc149bdd0f49db9423c9ba867133ce (127.15.217.65:34917)
I20260810 07:14:47.686318 17470 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.65@KRBTEST.COM'} at 127.15.217.65:35889
I20260810 07:14:47.691532 17525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:47.685540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:55547 (local address 127.15.217.126:35039)
0810 07:14:47.685586 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:47.685588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:47.685703 (+ 115us) server_negotiation.cc:407] Connection header received
0810 07:14:47.685768 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:47.685769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:47.685780 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:47.685834 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:47.686197 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:47.686828 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:47.687622 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:47.687825 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:47.687901 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:47.688923 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:47.688924 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:47.688925 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:47.688931 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:47.690334 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:47.690702 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:47.690703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:47.690703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:47.690759 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:47.690987 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:47.690988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:47.690988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:47.691106 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:47.691182 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:47.691345 (+ 163us) server_negotiation.cc:299] Negotiation successful
0810 07:14:47.691429 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 07:14:47.691533 17707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:47.685492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35039 (local address 127.15.217.65:55547)
0810 07:14:47.685627 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:47.685637 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:47.685718 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:47.685875 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:47.685876 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:47.686060 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:14:47.686114 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:47.686116 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:47.686928 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:47.686929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:47.687525 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:47.687527 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:47.687899 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:47.687900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:47.687989 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:47.688521 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:14:47.688527 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:14:47.688814 (+ 287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:14:47.690448 (+ 1634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:47.690449 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:47.690449 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:47.690624 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:47.690842 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:47.690843 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:47.690843 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:47.690908 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:47.691185 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:14:47.691186 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:14:47.691234 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 07:14:47.691319 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 07:14:47.691429 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":50,"thread_start_us":71,"threads_started":1}
WARNING: no policy specified for kudu/127.15.217.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab.
Entry for principal kudu/127.15.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab.
I20260810 07:14:47.712296 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.66:0
--local_ip_for_outbound_sockets=127.15.217.66
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:35039
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:40439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:14:47.799610 17709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:14:47.799876 17709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:14:47.799922 17709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:14:47.801203 17709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:14:47.801270 17709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:14:47.801306 17709 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 07:14:47.801347 17709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.66
I20260810 07:14:47.802822 17709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:40439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.217.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.217.126:35039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.66
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:14:47.803112 17709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:14:47.803372 17709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:14:47.808605 17715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:47.808619 17717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:47.808616 17714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:14:47.808914 17709 server_base.cc:1034] running on GCE node
I20260810 07:14:47.809140 17709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:14:47.809394 17709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:14:47.810549 17709 hybrid_clock.cc:648] HybridClock initialized: now 1786346087810536 us; error 29 us; skew 500 ppm
Aug 10 07:14:47 dist-test-slave-10pc krb5kdc[17418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346087, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:14:47.813388 17709 init.cc:377] Logged in from keytab as kudu/127.15.217.66@KRBTEST.COM (short username kudu)
I20260810 07:14:47.813855 17709 webserver.cc:460] Webserver started at http://127.15.217.66:42441/ using document root <none> and password file <none>
I20260810 07:14:47.814118 17709 fs_manager.cc:359] Metadata directory not provided
I20260810 07:14:47.814169 17709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:14:47.814318 17709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:14:47.815433 17709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-1/data/instance:
uuid: "c87d93dff03f4a8dac67eb164d56274a"
format_stamp: "Formatted at 2026-08-10 07:14:47 on dist-test-slave-10pc"
server_key: "0fe89b37cf1a1fe77207a75a7fcb219d"
server_key_iv: "4e134ac1e2cbe23cc099ffa4438faa48"
server_key_version: "encryptionkey@0"
I20260810 07:14:47.815851 17709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance:
uuid: "c87d93dff03f4a8dac67eb164d56274a"
format_stamp: "Formatted at 2026-08-10 07:14:47 on dist-test-slave-10pc"
server_key: "0fe89b37cf1a1fe77207a75a7fcb219d"
server_key_iv: "4e134ac1e2cbe23cc099ffa4438faa48"
server_key_version: "encryptionkey@0"
I20260810 07:14:47.817435 17709 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260810 07:14:47.818367 17726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:47.818583 17709 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 07:14:47.818683 17709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-1/wal
uuid: "c87d93dff03f4a8dac67eb164d56274a"
format_stamp: "Formatted at 2026-08-10 07:14:47 on dist-test-slave-10pc"
server_key: "0fe89b37cf1a1fe77207a75a7fcb219d"
server_key_iv: "4e134ac1e2cbe23cc099ffa4438faa48"
server_key_version: "encryptionkey@0"
I20260810 07:14:47.818742 17709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:14:47.838142 17709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:14:47.840657 17709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:14:47.840823 17709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:14:47.841423 17709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:14:47.841468 17709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:47.841492 17709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:14:47.841511 17709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:47.848834 17709 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.66:41727
I20260810 07:14:47.849006 17853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.66:41727 every 8 connection(s)
I20260810 07:14:47.849298 17709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
Aug 10 07:14:47 dist-test-slave-10pc krb5kdc[17418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346087, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:14:47.852839 17525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:47.842087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:34583 (local address 127.15.217.126:35039)
0810 07:14:47.842163 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:47.842165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:47.842414 (+ 249us) server_negotiation.cc:407] Connection header received
0810 07:14:47.843037 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:47.843039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:47.843055 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:47.843132 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:47.844160 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:47.845072 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:47.846752 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:47.847081 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:47.847164 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:47.850104 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:47.850106 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:47.850106 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:47.850113 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:47.851421 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:47.851837 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:47.851838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:47.851838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:47.851894 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:47.852143 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:47.852143 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:47.852143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:47.852254 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:47.852311 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:47.852701 (+ 390us) server_negotiation.cc:299] Negotiation successful
0810 07:14:47.852772 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:14:47.853052 17736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:47.842078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35039 (local address 127.15.217.66:34583)
0810 07:14:47.842283 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:47.842322 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:47.842866 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:47.843254 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:47.843260 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:47.843663 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:14:47.844035 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:47.844042 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:47.845200 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:47.845201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:47.846651 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:47.846655 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:47.847188 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:47.847189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:47.847356 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:47.848091 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:14:47.848102 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:14:47.850009 (+ 1907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:14:47.851534 (+ 1525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:47.851542 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:47.851544 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:47.851769 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:47.851986 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:47.851986 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:47.851986 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:47.852087 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:47.852302 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:14:47.852305 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:14:47.852425 (+ 120us) client_negotiation.cc:769] Sending connection context
0810 07:14:47.852685 (+ 260us) client_negotiation.cc:240] Negotiation successful
0810 07:14:47.852824 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":108,"threads_started":1}
I20260810 07:14:47.853612 17854 heartbeater.cc:347] Connected to a master server at 127.15.217.126:35039
I20260810 07:14:47.853726 17854 heartbeater.cc:464] Registering TS with master...
I20260810 07:14:47.853924 17854 heartbeater.cc:511] Master 127.15.217.126:35039 requested a full tablet report, sending...
I20260810 07:14:47.854385 17470 ts_manager.cc:194] Registered new tserver with Master: c87d93dff03f4a8dac67eb164d56274a (127.15.217.66:41727)
I20260810 07:14:47.854911 17470 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.66@KRBTEST.COM'} at 127.15.217.66:34583
I20260810 07:14:47.858819 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 17709
I20260810 07:14:47.858994 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance
I20260810 07:14:47.859216 16229 external_mini_cluster.cc:1428] Setting key 25c2b11de53035cd582d8d7055e10bb7
I20260810 07:14:47.861478 17860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:47.854717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35039 (local address 127.15.217.66:49281)
0810 07:14:47.854872 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:47.854882 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:47.854939 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:47.855096 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:47.855097 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:47.855336 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:14:47.855396 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:47.855398 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:47.856141 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:47.856141 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:47.856712 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:47.856714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:47.857055 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:47.857057 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:47.857131 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:47.857734 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:14:47.857740 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:14:47.858125 (+ 385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:14:47.859811 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:47.859813 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:47.859813 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:47.860120 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:47.860562 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:47.860563 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:47.860563 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:47.860639 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:47.861079 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:14:47.861080 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:14:47.861136 (+ 56us) client_negotiation.cc:769] Sending connection context
0810 07:14:47.861223 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 07:14:47.861395 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":82,"threads_started":1}
I20260810 07:14:47.861486 17525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:47.854803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:49281 (local address 127.15.217.126:35039)
0810 07:14:47.854848 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:47.854850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:47.854950 (+ 100us) server_negotiation.cc:407] Connection header received
0810 07:14:47.854976 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:47.854977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:47.854986 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:47.855046 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:47.855481 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:47.856050 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:47.856781 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:47.856958 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:47.857012 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:47.858205 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:47.858206 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:47.858206 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:47.858211 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:47.859730 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:47.860217 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:47.860219 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:47.860219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:47.860329 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:47.860717 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:47.860718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:47.860718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:47.860939 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:47.861022 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:47.861257 (+ 235us) server_negotiation.cc:299] Negotiation successful
0810 07:14:47.861347 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
WARNING: no policy specified for kudu/127.15.217.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab.
Entry for principal kudu/127.15.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab.
I20260810 07:14:47.894954 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.67:0
--local_ip_for_outbound_sockets=127.15.217.67
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:35039
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:40439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:14:47.982769 17863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:14:47.983062 17863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:14:47.983091 17863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:14:47.984623 17863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:14:47.984686 17863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:14:47.984709 17863 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 07:14:47.984762 17863 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.67
I20260810 07:14:47.986352 17863 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:40439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.217.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.217.126:35039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.67
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:14:47.986632 17863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:14:47.986899 17863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:14:47.992220 17869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:47.992223 17870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:14:47.992298 17863 server_base.cc:1034] running on GCE node
W20260810 07:14:47.992216 17872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:14:47.992758 17863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:14:47.993055 17863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:14:47.994236 17863 hybrid_clock.cc:648] HybridClock initialized: now 1786346087994211 us; error 36 us; skew 500 ppm
Aug 10 07:14:47 dist-test-slave-10pc krb5kdc[17418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346087, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:14:47.996847 17863 init.cc:377] Logged in from keytab as kudu/127.15.217.67@KRBTEST.COM (short username kudu)
I20260810 07:14:47.997406 17863 webserver.cc:460] Webserver started at http://127.15.217.67:33603/ using document root <none> and password file <none>
I20260810 07:14:47.997660 17863 fs_manager.cc:359] Metadata directory not provided
I20260810 07:14:47.997714 17863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:14:47.997869 17863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:14:47.999074 17863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-2/data/instance:
uuid: "b5be665e1ad04507887cec278a5efbfd"
format_stamp: "Formatted at 2026-08-10 07:14:47 on dist-test-slave-10pc"
server_key: "2720f05a19283ba2eafecb3c2107a0dd"
server_key_iv: "c6c8f1abc7e2f3b93742747011172e1c"
server_key_version: "encryptionkey@0"
I20260810 07:14:47.999492 17863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance:
uuid: "b5be665e1ad04507887cec278a5efbfd"
format_stamp: "Formatted at 2026-08-10 07:14:47 on dist-test-slave-10pc"
server_key: "2720f05a19283ba2eafecb3c2107a0dd"
server_key_iv: "c6c8f1abc7e2f3b93742747011172e1c"
server_key_version: "encryptionkey@0"
I20260810 07:14:48.000946 17863 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:14:48.001690 17879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:48.001866 17863 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 07:14:48.001974 17863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-2/wal
uuid: "b5be665e1ad04507887cec278a5efbfd"
format_stamp: "Formatted at 2026-08-10 07:14:47 on dist-test-slave-10pc"
server_key: "2720f05a19283ba2eafecb3c2107a0dd"
server_key_iv: "c6c8f1abc7e2f3b93742747011172e1c"
server_key_version: "encryptionkey@0"
I20260810 07:14:48.002053 17863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:14:48.024205 17863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:14:48.026523 17863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:14:48.026727 17863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:14:48.027427 17863 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:14:48.027482 17863 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:48.027513 17863 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:14:48.027537 17863 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 07:14:48 dist-test-slave-10pc krb5kdc[17418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346087, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:14:48.035341 17863 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.67:42441
I20260810 07:14:48.035784 18005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.67:42441 every 8 connection(s)
I20260810 07:14:48.035840 17863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
I20260810 07:14:48.038273 17525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:48.027863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:45581 (local address 127.15.217.126:35039)
0810 07:14:48.027934 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:48.027936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:48.028178 (+ 242us) server_negotiation.cc:407] Connection header received
0810 07:14:48.028840 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:48.028841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:48.028857 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:48.028919 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:48.029679 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.030194 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.031421 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.031706 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.031790 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:48.034758 (+ 2968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:48.034761 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:48.034762 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:48.034775 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:48.036665 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:48.037160 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:48.037161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:48.037161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:48.037230 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:48.037497 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:48.037498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:48.037498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:48.037635 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:48.037697 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:48.038109 (+ 412us) server_negotiation.cc:299] Negotiation successful
0810 07:14:48.038205 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 07:14:48.038457 17888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:48.027848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35039 (local address 127.15.217.67:45581)
0810 07:14:48.028018 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:48.028062 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:48.028723 (+ 661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:48.029011 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:48.029015 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:48.029359 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:14:48.029571 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:48.029576 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.030309 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.030310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:48.031325 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:48.031328 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.031800 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.031801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:48.031943 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:48.032645 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:14:48.032657 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:14:48.034636 (+ 1979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:14:48.036775 (+ 2139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:48.036784 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:48.036786 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:48.037072 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:48.037307 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:48.037308 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:48.037308 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:48.037428 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:48.037741 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:14:48.037749 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:14:48.037917 (+ 168us) client_negotiation.cc:769] Sending connection context
0810 07:14:48.038041 (+ 124us) client_negotiation.cc:240] Negotiation successful
0810 07:14:48.038197 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
I20260810 07:14:48.039211 18006 heartbeater.cc:347] Connected to a master server at 127.15.217.126:35039
I20260810 07:14:48.039342 18006 heartbeater.cc:464] Registering TS with master...
I20260810 07:14:48.039563 18006 heartbeater.cc:511] Master 127.15.217.126:35039 requested a full tablet report, sending...
I20260810 07:14:48.040019 17472 ts_manager.cc:194] Registered new tserver with Master: b5be665e1ad04507887cec278a5efbfd (127.15.217.67:42441)
I20260810 07:14:48.040433 17472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.67@KRBTEST.COM'} at 127.15.217.67:45581
I20260810 07:14:48.041291 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 17863
I20260810 07:14:48.041383 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance
I20260810 07:14:48.041575 16229 external_mini_cluster.cc:1428] Setting key 0d0ada7033021188c0d4e1160b2d8af7
I20260810 07:14:48.042526 16229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:14:48.047883 18012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:48.040681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35039 (local address 127.15.217.67:48235)
0810 07:14:48.040829 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:48.040840 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:48.040924 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:48.041062 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:48.041063 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:48.041261 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:14:48.041338 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:48.041340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.042327 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.042328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:48.043219 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:48.043222 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.043612 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.043613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:48.043713 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:48.044216 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:14:48.044223 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:14:48.044742 (+ 519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:14:48.046779 (+ 2037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:48.046780 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:48.046780 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:48.046982 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:48.047215 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:48.047216 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:48.047216 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:48.047279 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:48.047581 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:14:48.047581 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:14:48.047627 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 07:14:48.047689 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 07:14:48.047790 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":93,"threads_started":1}
I20260810 07:14:48.048159 17525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:48.040665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:48235 (local address 127.15.217.126:35039)
0810 07:14:48.040718 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:48.040721 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:48.040918 (+ 197us) server_negotiation.cc:407] Connection header received
0810 07:14:48.040985 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:48.040986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:48.040998 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:48.041059 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:48.041431 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.042225 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.043331 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.043526 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.043595 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:48.044852 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:48.044853 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:48.044853 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:48.044860 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:48.046673 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:48.047063 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:48.047064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:48.047065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:48.047136 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:48.047346 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:48.047347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:48.047348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:48.047511 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:48.047586 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:48.047737 (+ 151us) server_negotiation.cc:299] Negotiation successful
0810 07:14:48.047832 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
Aug 10 07:14:48 dist-test-slave-10pc krb5kdc[17418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346088, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 07:14:48 dist-test-slave-10pc krb5kdc[17418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346088, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:14:48.070724 17525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:48.062943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52130 (local address 127.15.217.126:35039)
0810 07:14:48.063016 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:48.063018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:48.063027 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:14:48.063057 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:48.063058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:48.063068 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:48.063119 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:48.063737 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.064325 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.065239 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.065449 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.065533 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:48.067881 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:48.067883 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:48.067883 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:48.067891 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:48.069352 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:48.069759 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:48.069760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:48.069760 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:48.069825 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:48.070095 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:48.070095 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:48.070096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:48.070227 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:48.070313 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:48.070515 (+ 202us) server_negotiation.cc:299] Negotiation successful
0810 07:14:48.070597 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 07:14:48.073063 17472 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52130:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:14:48.074707 17472 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:14:48.081810 18029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:48.078092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49574 (local address 127.15.217.65:34917)
0810 07:14:48.078457 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:48.078462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:48.078470 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:14:48.078508 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:48.078510 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:48.078529 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:48.078629 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:14:48.078728 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.079514 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.080847 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.081344 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.081434 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:48.081497 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:48.081658 (+ 161us) server_negotiation.cc:299] Negotiation successful
0810 07:14:48.081699 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":94,"threads_started":1}
I20260810 07:14:48.081799 18027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:48.077550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:34917 (local address 127.0.0.1:49574)
0810 07:14:48.078285 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:48.078296 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:48.078338 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:48.078630 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:48.078630 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:48.078632 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:14:48.078671 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:48.078673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.079620 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.079621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:48.080786 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:48.080788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.081446 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.081447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:48.081527 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:48.081541 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 07:14:48.081641 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 07:14:48.081698 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":712,"mutex_wait_us":32,"thread_start_us":102,"threads_started":1}
I20260810 07:14:48.082038 18026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:48.077480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58176 (local address 127.15.217.67:42441)
0810 07:14:48.077718 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:48.077723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:48.077734 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:14:48.077777 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:48.077779 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:48.077794 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:48.077898 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:14:48.078097 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.079202 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.080782 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.081466 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.081543 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:48.081600 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:48.081866 (+ 266us) server_negotiation.cc:299] Negotiation successful
0810 07:14:48.081916 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":116,"threads_started":1}
I20260810 07:14:48.082065 18025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:48.077354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:41727 (local address 127.0.0.1:42398)
0810 07:14:48.077522 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:48.077569 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:48.077640 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:48.078314 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:48.078315 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:48.078317 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:14:48.078381 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:48.078383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.079596 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.079597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:48.080629 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:48.080632 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.081643 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.081643 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:48.081711 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:48.081718 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 07:14:48.081772 (+ 54us) client_negotiation.cc:240] Negotiation successful
0810 07:14:48.081814 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":18,"thread_start_us":98,"threads_started":1}
I20260810 07:14:48.082228 18024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:48.077354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:42441 (local address 127.0.0.1:58176)
0810 07:14:48.077523 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:48.077568 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:48.077639 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:48.077905 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:48.077908 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:48.077910 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:14:48.077987 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:48.078000 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.079297 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.079298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:48.080691 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:48.080694 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.081640 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.081642 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:48.081730 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:48.081734 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:14:48.081781 (+ 47us) client_negotiation.cc:240] Negotiation successful
0810 07:14:48.081815 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":27,"thread_start_us":53,"threads_started":1}
I20260810 07:14:48.082593 17622 tablet_service.cc:1467] Processing CreateTablet for tablet 5ce9d8e00a22437a914d1dbe182f0283 (DEFAULT_TABLE table=test-table [id=7a09ca52ae9d42c9b38fe9f54c032847]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:14:48.082954 18028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:48.077741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42398 (local address 127.15.217.66:41727)
0810 07:14:48.078064 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:48.078070 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:48.078081 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:14:48.078133 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:48.078135 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:48.078155 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:48.078237 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:14:48.078478 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.079503 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.080722 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.081472 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.081543 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:48.081599 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:48.082780 (+ 1181us) server_negotiation.cc:299] Negotiation successful
0810 07:14:48.082856 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":109,"threads_started":1}
I20260810 07:14:48.083009 17622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ce9d8e00a22437a914d1dbe182f0283. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:14:48.082978 17933 tablet_service.cc:1467] Processing CreateTablet for tablet 5ce9d8e00a22437a914d1dbe182f0283 (DEFAULT_TABLE table=test-table [id=7a09ca52ae9d42c9b38fe9f54c032847]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:14:48.083315 17933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ce9d8e00a22437a914d1dbe182f0283. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:14:48.083565 17784 tablet_service.cc:1467] Processing CreateTablet for tablet 5ce9d8e00a22437a914d1dbe182f0283 (DEFAULT_TABLE table=test-table [id=7a09ca52ae9d42c9b38fe9f54c032847]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:14:48.083938 17784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ce9d8e00a22437a914d1dbe182f0283. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:14:48.086766 18030 tablet_bootstrap.cc:492] T 5ce9d8e00a22437a914d1dbe182f0283 P b5be665e1ad04507887cec278a5efbfd: Bootstrap starting.
I20260810 07:14:48.087146 18031 tablet_bootstrap.cc:492] T 5ce9d8e00a22437a914d1dbe182f0283 P 29fc149bdd0f49db9423c9ba867133ce: Bootstrap starting.
I20260810 07:14:48.087285 18032 tablet_bootstrap.cc:492] T 5ce9d8e00a22437a914d1dbe182f0283 P c87d93dff03f4a8dac67eb164d56274a: Bootstrap starting.
I20260810 07:14:48.087873 18030 tablet_bootstrap.cc:654] T 5ce9d8e00a22437a914d1dbe182f0283 P b5be665e1ad04507887cec278a5efbfd: Neither blocks nor log segments found. Creating new log.
I20260810 07:14:48.088042 18031 tablet_bootstrap.cc:654] T 5ce9d8e00a22437a914d1dbe182f0283 P 29fc149bdd0f49db9423c9ba867133ce: Neither blocks nor log segments found. Creating new log.
I20260810 07:14:48.088303 18030 log.cc:824] T 5ce9d8e00a22437a914d1dbe182f0283 P b5be665e1ad04507887cec278a5efbfd: Log is configured to *not* fsync() on all Append() calls
I20260810 07:14:48.088512 18031 log.cc:824] T 5ce9d8e00a22437a914d1dbe182f0283 P 29fc149bdd0f49db9423c9ba867133ce: Log is configured to *not* fsync() on all Append() calls
I20260810 07:14:48.088730 18032 tablet_bootstrap.cc:654] T 5ce9d8e00a22437a914d1dbe182f0283 P c87d93dff03f4a8dac67eb164d56274a: Neither blocks nor log segments found. Creating new log.
I20260810 07:14:48.089172 18032 log.cc:824] T 5ce9d8e00a22437a914d1dbe182f0283 P c87d93dff03f4a8dac67eb164d56274a: Log is configured to *not* fsync() on all Append() calls
I20260810 07:14:48.089577 18030 tablet_bootstrap.cc:492] T 5ce9d8e00a22437a914d1dbe182f0283 P b5be665e1ad04507887cec278a5efbfd: No bootstrap required, opened a new log
I20260810 07:14:48.089649 18031 tablet_bootstrap.cc:492] T 5ce9d8e00a22437a914d1dbe182f0283 P 29fc149bdd0f49db9423c9ba867133ce: No bootstrap required, opened a new log
I20260810 07:14:48.089668 18030 ts_tablet_manager.cc:1397] T 5ce9d8e00a22437a914d1dbe182f0283 P b5be665e1ad04507887cec278a5efbfd: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:14:48.089730 18031 ts_tablet_manager.cc:1397] T 5ce9d8e00a22437a914d1dbe182f0283 P 29fc149bdd0f49db9423c9ba867133ce: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:14:48.090031 18032 tablet_bootstrap.cc:492] T 5ce9d8e00a22437a914d1dbe182f0283 P c87d93dff03f4a8dac67eb164d56274a: No bootstrap required, opened a new log
I20260810 07:14:48.090102 18032 ts_tablet_manager.cc:1397] T 5ce9d8e00a22437a914d1dbe182f0283 P c87d93dff03f4a8dac67eb164d56274a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:14:48.091950 18030 raft_consensus.cc:348] T 5ce9d8e00a22437a914d1dbe182f0283 P b5be665e1ad04507887cec278a5efbfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } } peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } }
I20260810 07:14:48.092157 18030 raft_consensus.cc:374] T 5ce9d8e00a22437a914d1dbe182f0283 P b5be665e1ad04507887cec278a5efbfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:14:48.092222 18030 raft_consensus.cc:729] T 5ce9d8e00a22437a914d1dbe182f0283 P b5be665e1ad04507887cec278a5efbfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5be665e1ad04507887cec278a5efbfd, State: Initialized, Role: FOLLOWER
I20260810 07:14:48.092374 18030 consensus_queue.cc:260] T 5ce9d8e00a22437a914d1dbe182f0283 P b5be665e1ad04507887cec278a5efbfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } } peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } }
I20260810 07:14:48.092269 18031 raft_consensus.cc:348] T 5ce9d8e00a22437a914d1dbe182f0283 P 29fc149bdd0f49db9423c9ba867133ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } } peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } }
I20260810 07:14:48.092525 18031 raft_consensus.cc:374] T 5ce9d8e00a22437a914d1dbe182f0283 P 29fc149bdd0f49db9423c9ba867133ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:14:48.092597 18031 raft_consensus.cc:729] T 5ce9d8e00a22437a914d1dbe182f0283 P 29fc149bdd0f49db9423c9ba867133ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29fc149bdd0f49db9423c9ba867133ce, State: Initialized, Role: FOLLOWER
I20260810 07:14:48.092721 18031 consensus_queue.cc:260] T 5ce9d8e00a22437a914d1dbe182f0283 P 29fc149bdd0f49db9423c9ba867133ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } } peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } }
I20260810 07:14:48.092916 18030 ts_tablet_manager.cc:1428] T 5ce9d8e00a22437a914d1dbe182f0283 P b5be665e1ad04507887cec278a5efbfd: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260810 07:14:48.093000 18006 heartbeater.cc:503] Master 127.15.217.126:35039 was elected leader, sending a full tablet report...
I20260810 07:14:48.093318 18032 raft_consensus.cc:348] T 5ce9d8e00a22437a914d1dbe182f0283 P c87d93dff03f4a8dac67eb164d56274a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } } peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } }
I20260810 07:14:48.093503 18032 raft_consensus.cc:374] T 5ce9d8e00a22437a914d1dbe182f0283 P c87d93dff03f4a8dac67eb164d56274a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:14:48.093536 18032 raft_consensus.cc:729] T 5ce9d8e00a22437a914d1dbe182f0283 P c87d93dff03f4a8dac67eb164d56274a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c87d93dff03f4a8dac67eb164d56274a, State: Initialized, Role: FOLLOWER
I20260810 07:14:48.093653 18032 consensus_queue.cc:260] T 5ce9d8e00a22437a914d1dbe182f0283 P c87d93dff03f4a8dac67eb164d56274a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } } peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } }
I20260810 07:14:48.094009 18032 ts_tablet_manager.cc:1428] T 5ce9d8e00a22437a914d1dbe182f0283 P c87d93dff03f4a8dac67eb164d56274a: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260810 07:14:48.094009 18031 ts_tablet_manager.cc:1428] T 5ce9d8e00a22437a914d1dbe182f0283 P 29fc149bdd0f49db9423c9ba867133ce: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260810 07:14:48.094152 17700 heartbeater.cc:503] Master 127.15.217.126:35039 was elected leader, sending a full tablet report...
I20260810 07:14:48.094290 17854 heartbeater.cc:503] Master 127.15.217.126:35039 was elected leader, sending a full tablet report...
W20260810 07:14:48.100070 17855 tablet.cc:2369] T 5ce9d8e00a22437a914d1dbe182f0283 P c87d93dff03f4a8dac67eb164d56274a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:14:48.114027 18037 raft_consensus.cc:482] T 5ce9d8e00a22437a914d1dbe182f0283 P c87d93dff03f4a8dac67eb164d56274a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:14:48.114212 18037 raft_consensus.cc:504] T 5ce9d8e00a22437a914d1dbe182f0283 P c87d93dff03f4a8dac67eb164d56274a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } } peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } }
I20260810 07:14:48.114646 18037 leader_election.cc:288] T 5ce9d8e00a22437a914d1dbe182f0283 P c87d93dff03f4a8dac67eb164d56274a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5be665e1ad04507887cec278a5efbfd (127.15.217.67:42441), 29fc149bdd0f49db9423c9ba867133ce (127.15.217.65:34917)
I20260810 07:14:48.119092 18029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:48.114988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:33907 (local address 127.15.217.65:34917)
0810 07:14:48.115042 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:48.115045 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:48.115346 (+ 301us) server_negotiation.cc:407] Connection header received
0810 07:14:48.115455 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:48.115457 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:48.115478 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:48.115545 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:14:48.115984 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.116661 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.118060 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.118678 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.118748 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:48.118757 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:48.118992 (+ 235us) server_negotiation.cc:299] Negotiation successful
0810 07:14:48.119031 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 07:14:48.119195 18039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:48.115018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:34917 (local address 127.15.217.66:33907)
0810 07:14:48.115287 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:48.115297 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:48.115374 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:48.115563 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:48.115564 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:48.115566 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:14:48.115891 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:48.115894 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.116742 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.116743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:48.117978 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:48.117980 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.118766 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.118767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:48.118902 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:48.118908 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 07:14:48.118976 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 07:14:48.119026 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":2,"thread_start_us":198,"threads_started":1}
I20260810 07:14:48.119796 17736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:48.114989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:42441 (local address 127.15.217.66:41181)
0810 07:14:48.115060 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:48.115073 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:48.115139 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:48.115349 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:48.115351 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:48.115353 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:14:48.115530 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:48.115533 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.116544 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.116546 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:48.117856 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:48.117859 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.118766 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.118767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:48.119621 (+ 854us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:48.119626 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:14:48.119698 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 07:14:48.119743 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":35}
I20260810 07:14:48.120014 18026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:48.115001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:41181 (local address 127.15.217.67:42441)
0810 07:14:48.115111 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:48.115114 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:48.115129 (+ 15us) server_negotiation.cc:407] Connection header received
0810 07:14:48.115243 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:48.115245 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:48.115256 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:48.115342 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:14:48.115614 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.116409 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.117967 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.118657 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.118748 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:48.118757 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:48.119798 (+ 1041us) server_negotiation.cc:299] Negotiation successful
0810 07:14:48.119871 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 07:14:48.120009 17645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce9d8e00a22437a914d1dbe182f0283" candidate_uuid: "c87d93dff03f4a8dac67eb164d56274a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29fc149bdd0f49db9423c9ba867133ce" is_pre_election: true
I20260810 07:14:48.120216 17645 raft_consensus.cc:2454] T 5ce9d8e00a22437a914d1dbe182f0283 P 29fc149bdd0f49db9423c9ba867133ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c87d93dff03f4a8dac67eb164d56274a in term 0.
I20260810 07:14:48.120395 17955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce9d8e00a22437a914d1dbe182f0283" candidate_uuid: "c87d93dff03f4a8dac67eb164d56274a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5be665e1ad04507887cec278a5efbfd" is_pre_election: true
I20260810 07:14:48.120586 17727 leader_election.cc:302] T 5ce9d8e00a22437a914d1dbe182f0283 P c87d93dff03f4a8dac67eb164d56274a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29fc149bdd0f49db9423c9ba867133ce, c87d93dff03f4a8dac67eb164d56274a; no voters:
I20260810 07:14:48.120622 17955 raft_consensus.cc:2454] T 5ce9d8e00a22437a914d1dbe182f0283 P b5be665e1ad04507887cec278a5efbfd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c87d93dff03f4a8dac67eb164d56274a in term 0.
I20260810 07:14:48.120756 18037 raft_consensus.cc:2781] T 5ce9d8e00a22437a914d1dbe182f0283 P c87d93dff03f4a8dac67eb164d56274a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:14:48.120817 18037 raft_consensus.cc:482] T 5ce9d8e00a22437a914d1dbe182f0283 P c87d93dff03f4a8dac67eb164d56274a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:14:48.120847 18037 raft_consensus.cc:3037] T 5ce9d8e00a22437a914d1dbe182f0283 P c87d93dff03f4a8dac67eb164d56274a [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:14:48.121980 18037 raft_consensus.cc:504] T 5ce9d8e00a22437a914d1dbe182f0283 P c87d93dff03f4a8dac67eb164d56274a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } } peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } }
I20260810 07:14:48.122156 18037 leader_election.cc:288] T 5ce9d8e00a22437a914d1dbe182f0283 P c87d93dff03f4a8dac67eb164d56274a [CANDIDATE]: Term 1 election: Requested vote from peers b5be665e1ad04507887cec278a5efbfd (127.15.217.67:42441), 29fc149bdd0f49db9423c9ba867133ce (127.15.217.65:34917)
I20260810 07:14:48.122284 17955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce9d8e00a22437a914d1dbe182f0283" candidate_uuid: "c87d93dff03f4a8dac67eb164d56274a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5be665e1ad04507887cec278a5efbfd"
I20260810 07:14:48.122370 17955 raft_consensus.cc:3037] T 5ce9d8e00a22437a914d1dbe182f0283 P b5be665e1ad04507887cec278a5efbfd [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:14:48.122406 17645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce9d8e00a22437a914d1dbe182f0283" candidate_uuid: "c87d93dff03f4a8dac67eb164d56274a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29fc149bdd0f49db9423c9ba867133ce"
I20260810 07:14:48.122488 17645 raft_consensus.cc:3037] T 5ce9d8e00a22437a914d1dbe182f0283 P 29fc149bdd0f49db9423c9ba867133ce [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:14:48.123811 17645 raft_consensus.cc:2454] T 5ce9d8e00a22437a914d1dbe182f0283 P 29fc149bdd0f49db9423c9ba867133ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c87d93dff03f4a8dac67eb164d56274a in term 1.
I20260810 07:14:48.123811 17955 raft_consensus.cc:2454] T 5ce9d8e00a22437a914d1dbe182f0283 P b5be665e1ad04507887cec278a5efbfd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c87d93dff03f4a8dac67eb164d56274a in term 1.
I20260810 07:14:48.124074 17731 leader_election.cc:302] T 5ce9d8e00a22437a914d1dbe182f0283 P c87d93dff03f4a8dac67eb164d56274a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5be665e1ad04507887cec278a5efbfd, c87d93dff03f4a8dac67eb164d56274a; no voters:
I20260810 07:14:48.124188 18037 raft_consensus.cc:2781] T 5ce9d8e00a22437a914d1dbe182f0283 P c87d93dff03f4a8dac67eb164d56274a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:14:48.124481 18037 raft_consensus.cc:686] T 5ce9d8e00a22437a914d1dbe182f0283 P c87d93dff03f4a8dac67eb164d56274a [term 1 LEADER]: Becoming Leader. State: Replica: c87d93dff03f4a8dac67eb164d56274a, State: Running, Role: LEADER
I20260810 07:14:48.124610 18037 consensus_queue.cc:237] T 5ce9d8e00a22437a914d1dbe182f0283 P c87d93dff03f4a8dac67eb164d56274a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } } peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } }
I20260810 07:14:48.125612 17470 catalog_manager.cc:5495] T 5ce9d8e00a22437a914d1dbe182f0283 P c87d93dff03f4a8dac67eb164d56274a reported cstate change: term changed from 0 to 1, leader changed from <none> to c87d93dff03f4a8dac67eb164d56274a (127.15.217.66). New cstate: current_term: 1 leader_uuid: "c87d93dff03f4a8dac67eb164d56274a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } health_report { overall_health: UNKNOWN } } }
I20260810 07:14:48.143885 18028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:48.141229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42400 (local address 127.15.217.66:41727)
0810 07:14:48.141279 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:48.141283 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:48.141298 (+ 15us) server_negotiation.cc:407] Connection header received
0810 07:14:48.141381 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:48.141382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:48.141401 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:48.141458 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:14:48.141674 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.142183 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.142937 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.143163 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.143218 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:48.143439 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:14:48.143549 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:14:48.143623 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:48.143672 (+ 49us) server_negotiation.cc:299] Negotiation successful
0810 07:14:48.143699 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 07:14:48.146250 17645 raft_consensus.cc:1261] T 5ce9d8e00a22437a914d1dbe182f0283 P 29fc149bdd0f49db9423c9ba867133ce [term 1 FOLLOWER]: Refusing update from remote peer c87d93dff03f4a8dac67eb164d56274a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:14:48.146291 17955 raft_consensus.cc:1261] T 5ce9d8e00a22437a914d1dbe182f0283 P b5be665e1ad04507887cec278a5efbfd [term 1 FOLLOWER]: Refusing update from remote peer c87d93dff03f4a8dac67eb164d56274a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:14:48.146742 18040 consensus_queue.cc:1035] T 5ce9d8e00a22437a914d1dbe182f0283 P c87d93dff03f4a8dac67eb164d56274a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:14:48.146896 18037 consensus_queue.cc:1035] T 5ce9d8e00a22437a914d1dbe182f0283 P c87d93dff03f4a8dac67eb164d56274a [LEADER]: Connected to new peer: Peer: permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:14:48.150386 18045 mvcc.cc:205] Tried to move back new op lower bound from 7316873577044156416 to 7316873576959115264. Current Snapshot: MvccSnapshot[applied={T|T < 7316873577044156416}]
I20260810 07:14:48.155460 17472 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:14:48.155653 17472 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:14:48.156695 17472 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 36a23f9d47b9445884702e5af00539f6: Sending DeleteTablet for 3 replicas of tablet 5ce9d8e00a22437a914d1dbe182f0283
I20260810 07:14:48.157229 17784 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ce9d8e00a22437a914d1dbe182f0283 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:14:48 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:42398
I20260810 07:14:48.157262 17622 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ce9d8e00a22437a914d1dbe182f0283 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:14:48 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:49574
I20260810 07:14:48.157230 17933 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ce9d8e00a22437a914d1dbe182f0283 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:14:48 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:58176
I20260810 07:14:48.157909 18056 tablet_replica.cc:330] T 5ce9d8e00a22437a914d1dbe182f0283 P 29fc149bdd0f49db9423c9ba867133ce: stopping tablet replica
I20260810 07:14:48.158059 18056 raft_consensus.cc:2229] T 5ce9d8e00a22437a914d1dbe182f0283 P 29fc149bdd0f49db9423c9ba867133ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:14:48.158079 17472 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52130:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:14:48.158290 17472 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:14:48.158574 18056 raft_consensus.cc:2258] T 5ce9d8e00a22437a914d1dbe182f0283 P 29fc149bdd0f49db9423c9ba867133ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:14:48.158938 18054 tablet_replica.cc:330] T 5ce9d8e00a22437a914d1dbe182f0283 P b5be665e1ad04507887cec278a5efbfd: stopping tablet replica
I20260810 07:14:48.159067 18054 raft_consensus.cc:2229] T 5ce9d8e00a22437a914d1dbe182f0283 P b5be665e1ad04507887cec278a5efbfd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:14:48.159265 18054 raft_consensus.cc:2258] T 5ce9d8e00a22437a914d1dbe182f0283 P b5be665e1ad04507887cec278a5efbfd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:14:48.159789 17933 tablet_service.cc:1467] Processing CreateTablet for tablet 7b5aa8959ffb4445b997eb5387b704cb (DEFAULT_TABLE table=test-table [id=dabec47624ab4cccb369f3051beb6eef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:14:48.159982 17933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b5aa8959ffb4445b997eb5387b704cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:14:48.159940 17784 tablet_service.cc:1467] Processing CreateTablet for tablet 7b5aa8959ffb4445b997eb5387b704cb (DEFAULT_TABLE table=test-table [id=dabec47624ab4cccb369f3051beb6eef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:14:48.160095 17622 tablet_service.cc:1467] Processing CreateTablet for tablet 7b5aa8959ffb4445b997eb5387b704cb (DEFAULT_TABLE table=test-table [id=dabec47624ab4cccb369f3051beb6eef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:14:48.160130 17784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b5aa8959ffb4445b997eb5387b704cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:14:48.160255 17622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b5aa8959ffb4445b997eb5387b704cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:14:48.162380 18031 tablet_bootstrap.cc:492] T 7b5aa8959ffb4445b997eb5387b704cb P 29fc149bdd0f49db9423c9ba867133ce: Bootstrap starting.
I20260810 07:14:48.162647 18030 tablet_bootstrap.cc:492] T 7b5aa8959ffb4445b997eb5387b704cb P b5be665e1ad04507887cec278a5efbfd: Bootstrap starting.
I20260810 07:14:48.162796 18032 tablet_bootstrap.cc:492] T 7b5aa8959ffb4445b997eb5387b704cb P c87d93dff03f4a8dac67eb164d56274a: Bootstrap starting.
I20260810 07:14:48.163602 18030 tablet_bootstrap.cc:654] T 7b5aa8959ffb4445b997eb5387b704cb P b5be665e1ad04507887cec278a5efbfd: Neither blocks nor log segments found. Creating new log.
I20260810 07:14:48.163678 18031 tablet_bootstrap.cc:654] T 7b5aa8959ffb4445b997eb5387b704cb P 29fc149bdd0f49db9423c9ba867133ce: Neither blocks nor log segments found. Creating new log.
I20260810 07:14:48.164029 18032 tablet_bootstrap.cc:654] T 7b5aa8959ffb4445b997eb5387b704cb P c87d93dff03f4a8dac67eb164d56274a: Neither blocks nor log segments found. Creating new log.
I20260810 07:14:48.164952 18030 tablet_bootstrap.cc:492] T 7b5aa8959ffb4445b997eb5387b704cb P b5be665e1ad04507887cec278a5efbfd: No bootstrap required, opened a new log
I20260810 07:14:48.165009 18030 ts_tablet_manager.cc:1397] T 7b5aa8959ffb4445b997eb5387b704cb P b5be665e1ad04507887cec278a5efbfd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260810 07:14:48.165364 18031 tablet_bootstrap.cc:492] T 7b5aa8959ffb4445b997eb5387b704cb P 29fc149bdd0f49db9423c9ba867133ce: No bootstrap required, opened a new log
I20260810 07:14:48.165326 18030 raft_consensus.cc:348] T 7b5aa8959ffb4445b997eb5387b704cb P b5be665e1ad04507887cec278a5efbfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } } peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } }
I20260810 07:14:48.165417 18031 ts_tablet_manager.cc:1397] T 7b5aa8959ffb4445b997eb5387b704cb P 29fc149bdd0f49db9423c9ba867133ce: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260810 07:14:48.165424 18030 raft_consensus.cc:374] T 7b5aa8959ffb4445b997eb5387b704cb P b5be665e1ad04507887cec278a5efbfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:14:48.165463 18030 raft_consensus.cc:729] T 7b5aa8959ffb4445b997eb5387b704cb P b5be665e1ad04507887cec278a5efbfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5be665e1ad04507887cec278a5efbfd, State: Initialized, Role: FOLLOWER
I20260810 07:14:48.165526 18030 consensus_queue.cc:260] T 7b5aa8959ffb4445b997eb5387b704cb P b5be665e1ad04507887cec278a5efbfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } } peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } }
I20260810 07:14:48.165666 18030 ts_tablet_manager.cc:1428] T 7b5aa8959ffb4445b997eb5387b704cb P b5be665e1ad04507887cec278a5efbfd: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260810 07:14:48.165904 18032 tablet_bootstrap.cc:492] T 7b5aa8959ffb4445b997eb5387b704cb P c87d93dff03f4a8dac67eb164d56274a: No bootstrap required, opened a new log
I20260810 07:14:48.165956 18032 ts_tablet_manager.cc:1397] T 7b5aa8959ffb4445b997eb5387b704cb P c87d93dff03f4a8dac67eb164d56274a: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.003s
I20260810 07:14:48.165917 18031 raft_consensus.cc:348] T 7b5aa8959ffb4445b997eb5387b704cb P 29fc149bdd0f49db9423c9ba867133ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } } peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } }
I20260810 07:14:48.165997 18031 raft_consensus.cc:374] T 7b5aa8959ffb4445b997eb5387b704cb P 29fc149bdd0f49db9423c9ba867133ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:14:48.166019 18031 raft_consensus.cc:729] T 7b5aa8959ffb4445b997eb5387b704cb P 29fc149bdd0f49db9423c9ba867133ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29fc149bdd0f49db9423c9ba867133ce, State: Initialized, Role: FOLLOWER
I20260810 07:14:48.166052 18054 ts_tablet_manager.cc:1905] T 5ce9d8e00a22437a914d1dbe182f0283 P b5be665e1ad04507887cec278a5efbfd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:14:48.166100 18031 consensus_queue.cc:260] T 7b5aa8959ffb4445b997eb5387b704cb P 29fc149bdd0f49db9423c9ba867133ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } } peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } }
I20260810 07:14:48.166380 18032 raft_consensus.cc:348] T 7b5aa8959ffb4445b997eb5387b704cb P c87d93dff03f4a8dac67eb164d56274a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } } peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } }
I20260810 07:14:48.166525 18032 raft_consensus.cc:374] T 7b5aa8959ffb4445b997eb5387b704cb P c87d93dff03f4a8dac67eb164d56274a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:14:48.166558 18032 raft_consensus.cc:729] T 7b5aa8959ffb4445b997eb5387b704cb P c87d93dff03f4a8dac67eb164d56274a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c87d93dff03f4a8dac67eb164d56274a, State: Initialized, Role: FOLLOWER
I20260810 07:14:48.166607 18032 consensus_queue.cc:260] T 7b5aa8959ffb4445b997eb5387b704cb P c87d93dff03f4a8dac67eb164d56274a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } } peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } }
I20260810 07:14:48.166780 18032 ts_tablet_manager.cc:1428] T 7b5aa8959ffb4445b997eb5387b704cb P c87d93dff03f4a8dac67eb164d56274a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260810 07:14:48.167038 18031 ts_tablet_manager.cc:1428] T 7b5aa8959ffb4445b997eb5387b704cb P 29fc149bdd0f49db9423c9ba867133ce: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260810 07:14:48.167539 18056 ts_tablet_manager.cc:1905] T 5ce9d8e00a22437a914d1dbe182f0283 P 29fc149bdd0f49db9423c9ba867133ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:14:48.170234 18055 tablet_replica.cc:330] T 5ce9d8e00a22437a914d1dbe182f0283 P c87d93dff03f4a8dac67eb164d56274a: stopping tablet replica
I20260810 07:14:48.170297 18056 ts_tablet_manager.cc:1918] T 5ce9d8e00a22437a914d1dbe182f0283 P 29fc149bdd0f49db9423c9ba867133ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:14:48.170408 18055 raft_consensus.cc:2229] T 5ce9d8e00a22437a914d1dbe182f0283 P c87d93dff03f4a8dac67eb164d56274a [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:14:48.170457 18056 log.cc:1196] T 5ce9d8e00a22437a914d1dbe182f0283 P 29fc149bdd0f49db9423c9ba867133ce: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-0/wal/wals/5ce9d8e00a22437a914d1dbe182f0283
I20260810 07:14:48.170585 18055 raft_consensus.cc:2258] T 5ce9d8e00a22437a914d1dbe182f0283 P c87d93dff03f4a8dac67eb164d56274a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:14:48.170804 18056 ts_tablet_manager.cc:1939] T 5ce9d8e00a22437a914d1dbe182f0283 P 29fc149bdd0f49db9423c9ba867133ce: Deleting consensus metadata
I20260810 07:14:48.170791 18054 ts_tablet_manager.cc:1918] T 5ce9d8e00a22437a914d1dbe182f0283 P b5be665e1ad04507887cec278a5efbfd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:14:48.170902 18054 log.cc:1196] T 5ce9d8e00a22437a914d1dbe182f0283 P b5be665e1ad04507887cec278a5efbfd: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-2/wal/wals/5ce9d8e00a22437a914d1dbe182f0283
I20260810 07:14:48.171226 18054 ts_tablet_manager.cc:1939] T 5ce9d8e00a22437a914d1dbe182f0283 P b5be665e1ad04507887cec278a5efbfd: Deleting consensus metadata
I20260810 07:14:48.171654 17452 catalog_manager.cc:4841] TS 29fc149bdd0f49db9423c9ba867133ce (127.15.217.65:34917): tablet 5ce9d8e00a22437a914d1dbe182f0283 (table test-table [id=7a09ca52ae9d42c9b38fe9f54c032847]) successfully deleted
I20260810 07:14:48.171717 17457 catalog_manager.cc:4841] TS b5be665e1ad04507887cec278a5efbfd (127.15.217.67:42441): tablet 5ce9d8e00a22437a914d1dbe182f0283 (table test-table [id=7a09ca52ae9d42c9b38fe9f54c032847]) successfully deleted
I20260810 07:14:48.172039 18055 ts_tablet_manager.cc:1905] T 5ce9d8e00a22437a914d1dbe182f0283 P c87d93dff03f4a8dac67eb164d56274a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:14:48.173996 18055 ts_tablet_manager.cc:1918] T 5ce9d8e00a22437a914d1dbe182f0283 P c87d93dff03f4a8dac67eb164d56274a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:14:48.174065 18055 log.cc:1196] T 5ce9d8e00a22437a914d1dbe182f0283 P c87d93dff03f4a8dac67eb164d56274a: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-1/wal/wals/5ce9d8e00a22437a914d1dbe182f0283
I20260810 07:14:48.174320 18055 ts_tablet_manager.cc:1939] T 5ce9d8e00a22437a914d1dbe182f0283 P c87d93dff03f4a8dac67eb164d56274a: Deleting consensus metadata
I20260810 07:14:48.174919 17452 catalog_manager.cc:4841] TS c87d93dff03f4a8dac67eb164d56274a (127.15.217.66:41727): tablet 5ce9d8e00a22437a914d1dbe182f0283 (table test-table [id=7a09ca52ae9d42c9b38fe9f54c032847]) successfully deleted
W20260810 07:14:48.182912 17701 tablet.cc:2369] T 7b5aa8959ffb4445b997eb5387b704cb P 29fc149bdd0f49db9423c9ba867133ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:14:48.286527 18007 tablet.cc:2369] T 7b5aa8959ffb4445b997eb5387b704cb P b5be665e1ad04507887cec278a5efbfd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:14:48.384430 18038 raft_consensus.cc:482] T 7b5aa8959ffb4445b997eb5387b704cb P 29fc149bdd0f49db9423c9ba867133ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:14:48.384610 18038 raft_consensus.cc:504] T 7b5aa8959ffb4445b997eb5387b704cb P 29fc149bdd0f49db9423c9ba867133ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } } peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } }
I20260810 07:14:48.384992 18038 leader_election.cc:288] T 7b5aa8959ffb4445b997eb5387b704cb P 29fc149bdd0f49db9423c9ba867133ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c87d93dff03f4a8dac67eb164d56274a (127.15.217.66:41727), b5be665e1ad04507887cec278a5efbfd (127.15.217.67:42441)
I20260810 07:14:48.389358 18028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:48.385280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:39195 (local address 127.15.217.66:41727)
0810 07:14:48.385345 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:48.385347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:48.385452 (+ 105us) server_negotiation.cc:407] Connection header received
0810 07:14:48.385575 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:48.385577 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:48.385611 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:48.385669 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:14:48.385917 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.386648 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.388293 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.388922 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.388993 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:48.389001 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:48.389216 (+ 215us) server_negotiation.cc:299] Negotiation successful
0810 07:14:48.389276 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 07:14:48.389358 18026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:48.385374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:59715 (local address 127.15.217.67:42441)
0810 07:14:48.385471 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:48.385473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:48.385482 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:14:48.385539 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:48.385540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:48.385551 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:48.385613 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:14:48.385847 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.386635 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.388293 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.388877 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.388962 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:48.388974 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:48.389232 (+ 258us) server_negotiation.cc:299] Negotiation successful
0810 07:14:48.389289 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 07:14:48.389364 18061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:48.385211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:41727 (local address 127.15.217.65:39195)
0810 07:14:48.385375 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:48.385386 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:48.385486 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:48.385690 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:48.385690 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:48.385691 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:14:48.385838 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:48.385840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.386738 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.386738 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:48.388182 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:48.388187 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.388983 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.388984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:48.389052 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:48.389057 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:14:48.389151 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 07:14:48.389219 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":65,"thread_start_us":88,"threads_started":1}
I20260810 07:14:48.389458 18062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:48.385187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:42441 (local address 127.15.217.65:59715)
0810 07:14:48.385350 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:48.385362 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:48.385426 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:48.385630 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:48.385631 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:48.385634 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:14:48.385739 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:48.385741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.386737 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.386738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:48.388190 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:48.388192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.388973 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.388974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:48.389133 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:48.389138 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:14:48.389193 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 07:14:48.389223 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":29,"thread_start_us":90,"threads_started":1}
I20260810 07:14:48.389926 17955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b5aa8959ffb4445b997eb5387b704cb" candidate_uuid: "29fc149bdd0f49db9423c9ba867133ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5be665e1ad04507887cec278a5efbfd" is_pre_election: true
I20260810 07:14:48.390054 17955 raft_consensus.cc:2454] T 7b5aa8959ffb4445b997eb5387b704cb P b5be665e1ad04507887cec278a5efbfd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29fc149bdd0f49db9423c9ba867133ce in term 0.
I20260810 07:14:48.389982 17804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b5aa8959ffb4445b997eb5387b704cb" candidate_uuid: "29fc149bdd0f49db9423c9ba867133ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c87d93dff03f4a8dac67eb164d56274a" is_pre_election: true
I20260810 07:14:48.390174 17804 raft_consensus.cc:2454] T 7b5aa8959ffb4445b997eb5387b704cb P c87d93dff03f4a8dac67eb164d56274a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29fc149bdd0f49db9423c9ba867133ce in term 0.
I20260810 07:14:48.390292 17571 leader_election.cc:302] T 7b5aa8959ffb4445b997eb5387b704cb P 29fc149bdd0f49db9423c9ba867133ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29fc149bdd0f49db9423c9ba867133ce, b5be665e1ad04507887cec278a5efbfd; no voters:
I20260810 07:14:48.390497 18038 raft_consensus.cc:2781] T 7b5aa8959ffb4445b997eb5387b704cb P 29fc149bdd0f49db9423c9ba867133ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:14:48.390544 18038 raft_consensus.cc:482] T 7b5aa8959ffb4445b997eb5387b704cb P 29fc149bdd0f49db9423c9ba867133ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:14:48.390559 18038 raft_consensus.cc:3037] T 7b5aa8959ffb4445b997eb5387b704cb P 29fc149bdd0f49db9423c9ba867133ce [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:14:48.391516 18038 raft_consensus.cc:504] T 7b5aa8959ffb4445b997eb5387b704cb P 29fc149bdd0f49db9423c9ba867133ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } } peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } }
I20260810 07:14:48.391687 18038 leader_election.cc:288] T 7b5aa8959ffb4445b997eb5387b704cb P 29fc149bdd0f49db9423c9ba867133ce [CANDIDATE]: Term 1 election: Requested vote from peers c87d93dff03f4a8dac67eb164d56274a (127.15.217.66:41727), b5be665e1ad04507887cec278a5efbfd (127.15.217.67:42441)
I20260810 07:14:48.391834 17804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b5aa8959ffb4445b997eb5387b704cb" candidate_uuid: "29fc149bdd0f49db9423c9ba867133ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c87d93dff03f4a8dac67eb164d56274a"
I20260810 07:14:48.391852 17955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b5aa8959ffb4445b997eb5387b704cb" candidate_uuid: "29fc149bdd0f49db9423c9ba867133ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5be665e1ad04507887cec278a5efbfd"
I20260810 07:14:48.391923 17955 raft_consensus.cc:3037] T 7b5aa8959ffb4445b997eb5387b704cb P b5be665e1ad04507887cec278a5efbfd [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:14:48.391923 17804 raft_consensus.cc:3037] T 7b5aa8959ffb4445b997eb5387b704cb P c87d93dff03f4a8dac67eb164d56274a [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:14:48.392773 17955 raft_consensus.cc:2454] T 7b5aa8959ffb4445b997eb5387b704cb P b5be665e1ad04507887cec278a5efbfd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29fc149bdd0f49db9423c9ba867133ce in term 1.
I20260810 07:14:48.392844 17804 raft_consensus.cc:2454] T 7b5aa8959ffb4445b997eb5387b704cb P c87d93dff03f4a8dac67eb164d56274a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29fc149bdd0f49db9423c9ba867133ce in term 1.
I20260810 07:14:48.392957 17571 leader_election.cc:302] T 7b5aa8959ffb4445b997eb5387b704cb P 29fc149bdd0f49db9423c9ba867133ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29fc149bdd0f49db9423c9ba867133ce, b5be665e1ad04507887cec278a5efbfd; no voters:
I20260810 07:14:48.393069 18038 raft_consensus.cc:2781] T 7b5aa8959ffb4445b997eb5387b704cb P 29fc149bdd0f49db9423c9ba867133ce [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:14:48.393211 18038 raft_consensus.cc:686] T 7b5aa8959ffb4445b997eb5387b704cb P 29fc149bdd0f49db9423c9ba867133ce [term 1 LEADER]: Becoming Leader. State: Replica: 29fc149bdd0f49db9423c9ba867133ce, State: Running, Role: LEADER
I20260810 07:14:48.393288 18038 consensus_queue.cc:237] T 7b5aa8959ffb4445b997eb5387b704cb P 29fc149bdd0f49db9423c9ba867133ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } } peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } }
I20260810 07:14:48.394038 17472 catalog_manager.cc:5495] T 7b5aa8959ffb4445b997eb5387b704cb P 29fc149bdd0f49db9423c9ba867133ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 29fc149bdd0f49db9423c9ba867133ce (127.15.217.65). New cstate: current_term: 1 leader_uuid: "29fc149bdd0f49db9423c9ba867133ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } health_report { overall_health: UNKNOWN } } }
Aug 10 07:14:48 dist-test-slave-10pc krb5kdc[17418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346087, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:14:48.472560 17525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:48.465397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52146 (local address 127.15.217.126:35039)
0810 07:14:48.465475 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:48.465477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:48.465749 (+ 272us) server_negotiation.cc:407] Connection header received
0810 07:14:48.465801 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:48.465802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:48.465813 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:48.465867 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:48.466383 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.466867 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.467588 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.467776 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.467826 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:48.469992 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:48.469993 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:48.469993 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:48.470007 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:48.471504 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:48.471867 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:48.471868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:48.471868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:48.471923 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:48.472091 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:48.472092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:48.472092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:48.472218 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:48.472270 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:48.472392 (+ 122us) server_negotiation.cc:299] Negotiation successful
0810 07:14:48.472489 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 07:14:48.472673 18070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:48.465394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35039 (local address 127.0.0.1:52146)
0810 07:14:48.465633 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:48.465643 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:48.465747 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:48.465888 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:48.465889 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:48.466258 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:14:48.466312 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:48.466315 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.466937 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.466938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:48.467511 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:48.467513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.467860 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.467861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:48.467905 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:48.468492 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:14:48.468498 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:14:48.469908 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:14:48.471587 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:48.471595 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:48.471598 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:48.471797 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:48.471981 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:48.471982 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:48.471982 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:48.472038 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:48.472273 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:14:48.472280 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:14:48.472324 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 07:14:48.472376 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 07:14:48.472562 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":80,"thread_start_us":129,"threads_started":1}
I20260810 07:14:48.474794 17472 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:52146:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 07:14:48.475057 17472 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:14:48.476809 17784 tablet_service.cc:1467] Processing CreateTablet for tablet 2501f4e728374f6898de00223a6a5460 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1c2bfee9408e49fcbb23102ece959882]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 07:14:48.476802 17622 tablet_service.cc:1467] Processing CreateTablet for tablet 2501f4e728374f6898de00223a6a5460 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1c2bfee9408e49fcbb23102ece959882]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 07:14:48.476819 17933 tablet_service.cc:1467] Processing CreateTablet for tablet 2501f4e728374f6898de00223a6a5460 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1c2bfee9408e49fcbb23102ece959882]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 07:14:48.476994 17622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2501f4e728374f6898de00223a6a5460. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:14:48.476994 17784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2501f4e728374f6898de00223a6a5460. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:14:48.477006 17933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2501f4e728374f6898de00223a6a5460. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:14:48.478556 18030 tablet_bootstrap.cc:492] T 2501f4e728374f6898de00223a6a5460 P b5be665e1ad04507887cec278a5efbfd: Bootstrap starting.
I20260810 07:14:48.478912 18032 tablet_bootstrap.cc:492] T 2501f4e728374f6898de00223a6a5460 P c87d93dff03f4a8dac67eb164d56274a: Bootstrap starting.
I20260810 07:14:48.478943 18031 tablet_bootstrap.cc:492] T 2501f4e728374f6898de00223a6a5460 P 29fc149bdd0f49db9423c9ba867133ce: Bootstrap starting.
I20260810 07:14:48.479862 18032 tablet_bootstrap.cc:654] T 2501f4e728374f6898de00223a6a5460 P c87d93dff03f4a8dac67eb164d56274a: Neither blocks nor log segments found. Creating new log.
I20260810 07:14:48.479938 18031 tablet_bootstrap.cc:654] T 2501f4e728374f6898de00223a6a5460 P 29fc149bdd0f49db9423c9ba867133ce: Neither blocks nor log segments found. Creating new log.
I20260810 07:14:48.479930 18030 tablet_bootstrap.cc:654] T 2501f4e728374f6898de00223a6a5460 P b5be665e1ad04507887cec278a5efbfd: Neither blocks nor log segments found. Creating new log.
I20260810 07:14:48.480991 18032 tablet_bootstrap.cc:492] T 2501f4e728374f6898de00223a6a5460 P c87d93dff03f4a8dac67eb164d56274a: No bootstrap required, opened a new log
I20260810 07:14:48.481038 18032 ts_tablet_manager.cc:1397] T 2501f4e728374f6898de00223a6a5460 P c87d93dff03f4a8dac67eb164d56274a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260810 07:14:48.481160 18032 raft_consensus.cc:348] T 2501f4e728374f6898de00223a6a5460 P c87d93dff03f4a8dac67eb164d56274a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } } peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } }
I20260810 07:14:48.481280 18032 raft_consensus.cc:374] T 2501f4e728374f6898de00223a6a5460 P c87d93dff03f4a8dac67eb164d56274a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:14:48.481313 18032 raft_consensus.cc:729] T 2501f4e728374f6898de00223a6a5460 P c87d93dff03f4a8dac67eb164d56274a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c87d93dff03f4a8dac67eb164d56274a, State: Initialized, Role: FOLLOWER
I20260810 07:14:48.481375 18032 consensus_queue.cc:260] T 2501f4e728374f6898de00223a6a5460 P c87d93dff03f4a8dac67eb164d56274a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } } peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } }
I20260810 07:14:48.481410 18031 tablet_bootstrap.cc:492] T 2501f4e728374f6898de00223a6a5460 P 29fc149bdd0f49db9423c9ba867133ce: No bootstrap required, opened a new log
I20260810 07:14:48.481456 18031 ts_tablet_manager.cc:1397] T 2501f4e728374f6898de00223a6a5460 P 29fc149bdd0f49db9423c9ba867133ce: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260810 07:14:48.481489 18032 ts_tablet_manager.cc:1428] T 2501f4e728374f6898de00223a6a5460 P c87d93dff03f4a8dac67eb164d56274a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:48.481876 18031 raft_consensus.cc:348] T 2501f4e728374f6898de00223a6a5460 P 29fc149bdd0f49db9423c9ba867133ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } } peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } }
I20260810 07:14:48.481977 18031 raft_consensus.cc:374] T 2501f4e728374f6898de00223a6a5460 P 29fc149bdd0f49db9423c9ba867133ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:14:48.482013 18031 raft_consensus.cc:729] T 2501f4e728374f6898de00223a6a5460 P 29fc149bdd0f49db9423c9ba867133ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29fc149bdd0f49db9423c9ba867133ce, State: Initialized, Role: FOLLOWER
I20260810 07:14:48.482067 18031 consensus_queue.cc:260] T 2501f4e728374f6898de00223a6a5460 P 29fc149bdd0f49db9423c9ba867133ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } } peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } }
I20260810 07:14:48.482172 18031 ts_tablet_manager.cc:1428] T 2501f4e728374f6898de00223a6a5460 P 29fc149bdd0f49db9423c9ba867133ce: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260810 07:14:48.482267 18038 tablet_replica.cc:439] T 2501f4e728374f6898de00223a6a5460 P 29fc149bdd0f49db9423c9ba867133ce: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } } peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } } }
I20260810 07:14:48.482321 18049 tablet_replica.cc:439] T 2501f4e728374f6898de00223a6a5460 P c87d93dff03f4a8dac67eb164d56274a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } } peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } } }
I20260810 07:14:48.482385 18038 tablet_replica.cc:442] T 2501f4e728374f6898de00223a6a5460 P 29fc149bdd0f49db9423c9ba867133ce: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:14:48.482409 18049 tablet_replica.cc:442] T 2501f4e728374f6898de00223a6a5460 P c87d93dff03f4a8dac67eb164d56274a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:14:48.482645 18030 tablet_bootstrap.cc:492] T 2501f4e728374f6898de00223a6a5460 P b5be665e1ad04507887cec278a5efbfd: No bootstrap required, opened a new log
I20260810 07:14:48.482710 18030 ts_tablet_manager.cc:1397] T 2501f4e728374f6898de00223a6a5460 P b5be665e1ad04507887cec278a5efbfd: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260810 07:14:48.483116 18030 raft_consensus.cc:348] T 2501f4e728374f6898de00223a6a5460 P b5be665e1ad04507887cec278a5efbfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } } peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } }
I20260810 07:14:48.483224 18030 raft_consensus.cc:374] T 2501f4e728374f6898de00223a6a5460 P b5be665e1ad04507887cec278a5efbfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:14:48.483253 18030 raft_consensus.cc:729] T 2501f4e728374f6898de00223a6a5460 P b5be665e1ad04507887cec278a5efbfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5be665e1ad04507887cec278a5efbfd, State: Initialized, Role: FOLLOWER
I20260810 07:14:48.483309 18030 consensus_queue.cc:260] T 2501f4e728374f6898de00223a6a5460 P b5be665e1ad04507887cec278a5efbfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } } peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } }
I20260810 07:14:48.483418 18030 ts_tablet_manager.cc:1428] T 2501f4e728374f6898de00223a6a5460 P b5be665e1ad04507887cec278a5efbfd: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 07:14:48.483566 18036 tablet_replica.cc:439] T 2501f4e728374f6898de00223a6a5460 P b5be665e1ad04507887cec278a5efbfd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } } peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } } }
I20260810 07:14:48.483721 18036 tablet_replica.cc:442] T 2501f4e728374f6898de00223a6a5460 P b5be665e1ad04507887cec278a5efbfd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:14:48.670629 18049 raft_consensus.cc:482] T 2501f4e728374f6898de00223a6a5460 P c87d93dff03f4a8dac67eb164d56274a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:14:48.670780 18049 raft_consensus.cc:504] T 2501f4e728374f6898de00223a6a5460 P c87d93dff03f4a8dac67eb164d56274a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } } peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } }
I20260810 07:14:48.671058 18049 leader_election.cc:288] T 2501f4e728374f6898de00223a6a5460 P c87d93dff03f4a8dac67eb164d56274a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29fc149bdd0f49db9423c9ba867133ce (127.15.217.65:34917), b5be665e1ad04507887cec278a5efbfd (127.15.217.67:42441)
I20260810 07:14:48.671325 17955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2501f4e728374f6898de00223a6a5460" candidate_uuid: "c87d93dff03f4a8dac67eb164d56274a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5be665e1ad04507887cec278a5efbfd" is_pre_election: true
I20260810 07:14:48.671427 17955 raft_consensus.cc:2454] T 2501f4e728374f6898de00223a6a5460 P b5be665e1ad04507887cec278a5efbfd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c87d93dff03f4a8dac67eb164d56274a in term 0.
I20260810 07:14:48.671360 17645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2501f4e728374f6898de00223a6a5460" candidate_uuid: "c87d93dff03f4a8dac67eb164d56274a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29fc149bdd0f49db9423c9ba867133ce" is_pre_election: true
I20260810 07:14:48.671541 17645 raft_consensus.cc:2454] T 2501f4e728374f6898de00223a6a5460 P 29fc149bdd0f49db9423c9ba867133ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c87d93dff03f4a8dac67eb164d56274a in term 0.
I20260810 07:14:48.671623 17731 leader_election.cc:302] T 2501f4e728374f6898de00223a6a5460 P c87d93dff03f4a8dac67eb164d56274a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5be665e1ad04507887cec278a5efbfd, c87d93dff03f4a8dac67eb164d56274a; no voters:
I20260810 07:14:48.671753 18049 raft_consensus.cc:2781] T 2501f4e728374f6898de00223a6a5460 P c87d93dff03f4a8dac67eb164d56274a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:14:48.671783 18049 raft_consensus.cc:482] T 2501f4e728374f6898de00223a6a5460 P c87d93dff03f4a8dac67eb164d56274a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:14:48.671801 18049 raft_consensus.cc:3037] T 2501f4e728374f6898de00223a6a5460 P c87d93dff03f4a8dac67eb164d56274a [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:14:48.672648 18049 raft_consensus.cc:504] T 2501f4e728374f6898de00223a6a5460 P c87d93dff03f4a8dac67eb164d56274a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } } peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } }
I20260810 07:14:48.672806 18049 leader_election.cc:288] T 2501f4e728374f6898de00223a6a5460 P c87d93dff03f4a8dac67eb164d56274a [CANDIDATE]: Term 1 election: Requested vote from peers 29fc149bdd0f49db9423c9ba867133ce (127.15.217.65:34917), b5be665e1ad04507887cec278a5efbfd (127.15.217.67:42441)
I20260810 07:14:48.672976 17645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2501f4e728374f6898de00223a6a5460" candidate_uuid: "c87d93dff03f4a8dac67eb164d56274a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29fc149bdd0f49db9423c9ba867133ce"
I20260810 07:14:48.673071 17645 raft_consensus.cc:3037] T 2501f4e728374f6898de00223a6a5460 P 29fc149bdd0f49db9423c9ba867133ce [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:14:48.673131 17955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2501f4e728374f6898de00223a6a5460" candidate_uuid: "c87d93dff03f4a8dac67eb164d56274a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5be665e1ad04507887cec278a5efbfd"
I20260810 07:14:48.673223 17955 raft_consensus.cc:3037] T 2501f4e728374f6898de00223a6a5460 P b5be665e1ad04507887cec278a5efbfd [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:14:48.673820 17645 raft_consensus.cc:2454] T 2501f4e728374f6898de00223a6a5460 P 29fc149bdd0f49db9423c9ba867133ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c87d93dff03f4a8dac67eb164d56274a in term 1.
I20260810 07:14:48.673928 17955 raft_consensus.cc:2454] T 2501f4e728374f6898de00223a6a5460 P b5be665e1ad04507887cec278a5efbfd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c87d93dff03f4a8dac67eb164d56274a in term 1.
I20260810 07:14:48.674080 17727 leader_election.cc:302] T 2501f4e728374f6898de00223a6a5460 P c87d93dff03f4a8dac67eb164d56274a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29fc149bdd0f49db9423c9ba867133ce, c87d93dff03f4a8dac67eb164d56274a; no voters:
I20260810 07:14:48.674194 18049 raft_consensus.cc:2781] T 2501f4e728374f6898de00223a6a5460 P c87d93dff03f4a8dac67eb164d56274a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:14:48.674306 18049 raft_consensus.cc:686] T 2501f4e728374f6898de00223a6a5460 P c87d93dff03f4a8dac67eb164d56274a [term 1 LEADER]: Becoming Leader. State: Replica: c87d93dff03f4a8dac67eb164d56274a, State: Running, Role: LEADER
I20260810 07:14:48.674351 18049 consensus_queue.cc:237] T 2501f4e728374f6898de00223a6a5460 P c87d93dff03f4a8dac67eb164d56274a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } } peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } }
I20260810 07:14:48.674681 18074 tablet_replica.cc:439] T 2501f4e728374f6898de00223a6a5460 P c87d93dff03f4a8dac67eb164d56274a: TxnStatusTablet state changed. Reason: New leader c87d93dff03f4a8dac67eb164d56274a. Latest consensus state: current_term: 1 leader_uuid: "c87d93dff03f4a8dac67eb164d56274a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } } peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } } }
I20260810 07:14:48.674783 18074 tablet_replica.cc:442] T 2501f4e728374f6898de00223a6a5460 P c87d93dff03f4a8dac67eb164d56274a: This TxnStatusTablet replica's current role is: LEADER
I20260810 07:14:48.675011 18076 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 07:14:48.675155 18076 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 07:14:48.675063 17472 catalog_manager.cc:5495] T 2501f4e728374f6898de00223a6a5460 P c87d93dff03f4a8dac67eb164d56274a reported cstate change: term changed from 0 to 1, leader changed from <none> to c87d93dff03f4a8dac67eb164d56274a (127.15.217.66). New cstate: current_term: 1 leader_uuid: "c87d93dff03f4a8dac67eb164d56274a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } health_report { overall_health: HEALTHY } } }
I20260810 07:14:48.709106 18028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:48.706318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42412 (local address 127.15.217.66:41727)
0810 07:14:48.706367 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:48.706369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:48.706409 (+ 40us) server_negotiation.cc:407] Connection header received
0810 07:14:48.706507 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:48.706507 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:48.706517 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:48.706560 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:14:48.706710 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.707185 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.708329 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.708616 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.708675 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:48.708894 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:14:48.708925 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:14:48.708964 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:48.709014 (+ 50us) server_negotiation.cc:299] Negotiation successful
0810 07:14:48.709034 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 07:14:48.709161 18070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:48.706270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:41727 (local address 127.0.0.1:42412)
0810 07:14:48.706345 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:48.706359 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:48.706423 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:48.706585 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:48.706586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:48.706588 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:14:48.706647 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:48.706650 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.707279 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.707280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:48.708222 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:48.708225 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.708704 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.708705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:48.708780 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:48.708800 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:14:48.708965 (+ 165us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:14:48.708970 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:14:48.709013 (+ 43us) client_negotiation.cc:240] Negotiation successful
0810 07:14:48.709071 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260810 07:14:48.710448 17955 raft_consensus.cc:1261] T 2501f4e728374f6898de00223a6a5460 P b5be665e1ad04507887cec278a5efbfd [term 1 FOLLOWER]: Refusing update from remote peer c87d93dff03f4a8dac67eb164d56274a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:14:48.710471 17645 raft_consensus.cc:1261] T 2501f4e728374f6898de00223a6a5460 P 29fc149bdd0f49db9423c9ba867133ce [term 1 FOLLOWER]: Refusing update from remote peer c87d93dff03f4a8dac67eb164d56274a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:14:48.710678 18074 consensus_queue.cc:1035] T 2501f4e728374f6898de00223a6a5460 P c87d93dff03f4a8dac67eb164d56274a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:14:48.710814 18049 consensus_queue.cc:1035] T 2501f4e728374f6898de00223a6a5460 P c87d93dff03f4a8dac67eb164d56274a [LEADER]: Connected to new peer: Peer: permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:14:48.711350 18036 tablet_replica.cc:439] T 2501f4e728374f6898de00223a6a5460 P b5be665e1ad04507887cec278a5efbfd: TxnStatusTablet state changed. Reason: New leader c87d93dff03f4a8dac67eb164d56274a. Latest consensus state: current_term: 1 leader_uuid: "c87d93dff03f4a8dac67eb164d56274a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } } peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } } }
I20260810 07:14:48.711449 18036 tablet_replica.cc:442] T 2501f4e728374f6898de00223a6a5460 P b5be665e1ad04507887cec278a5efbfd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:14:48.711535 18038 tablet_replica.cc:439] T 2501f4e728374f6898de00223a6a5460 P 29fc149bdd0f49db9423c9ba867133ce: TxnStatusTablet state changed. Reason: New leader c87d93dff03f4a8dac67eb164d56274a. Latest consensus state: current_term: 1 leader_uuid: "c87d93dff03f4a8dac67eb164d56274a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } } peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } } }
I20260810 07:14:48.711629 18038 tablet_replica.cc:442] T 2501f4e728374f6898de00223a6a5460 P 29fc149bdd0f49db9423c9ba867133ce: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:14:48.712611 18074 tablet_replica.cc:439] T 2501f4e728374f6898de00223a6a5460 P c87d93dff03f4a8dac67eb164d56274a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c87d93dff03f4a8dac67eb164d56274a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } } peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } } }
I20260810 07:14:48.712707 18074 tablet_replica.cc:442] T 2501f4e728374f6898de00223a6a5460 P c87d93dff03f4a8dac67eb164d56274a: This TxnStatusTablet replica's current role is: LEADER
I20260810 07:14:48.712661 18036 tablet_replica.cc:439] T 2501f4e728374f6898de00223a6a5460 P b5be665e1ad04507887cec278a5efbfd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c87d93dff03f4a8dac67eb164d56274a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } } peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } } }
I20260810 07:14:48.712739 18036 tablet_replica.cc:442] T 2501f4e728374f6898de00223a6a5460 P b5be665e1ad04507887cec278a5efbfd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:14:48.712809 18081 mvcc.cc:205] Tried to move back new op lower bound from 7316873579356073984 to 7316873579210698752. Current Snapshot: MvccSnapshot[applied={T|T < 7316873579356073984}]
I20260810 07:14:48.712879 18074 tablet_replica.cc:439] T 2501f4e728374f6898de00223a6a5460 P c87d93dff03f4a8dac67eb164d56274a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c87d93dff03f4a8dac67eb164d56274a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } } peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } } }
I20260810 07:14:48.712957 18074 tablet_replica.cc:442] T 2501f4e728374f6898de00223a6a5460 P c87d93dff03f4a8dac67eb164d56274a: This TxnStatusTablet replica's current role is: LEADER
I20260810 07:14:48.713304 18038 tablet_replica.cc:439] T 2501f4e728374f6898de00223a6a5460 P 29fc149bdd0f49db9423c9ba867133ce: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c87d93dff03f4a8dac67eb164d56274a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fc149bdd0f49db9423c9ba867133ce" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34917 } } peers { permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 } } peers { permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 } } }
I20260810 07:14:48.713387 18038 tablet_replica.cc:442] T 2501f4e728374f6898de00223a6a5460 P 29fc149bdd0f49db9423c9ba867133ce: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:14:48.714264 18082 mvcc.cc:205] Tried to move back new op lower bound from 7316873579356073984 to 7316873579210698752. Current Snapshot: MvccSnapshot[applied={T|T < 7316873579356073984 or (T in {7316873579356073984})}]
I20260810 07:14:48.718839 18091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:48.715856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49584 (local address 127.15.217.65:34917)
0810 07:14:48.715950 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:48.715953 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:48.715968 (+ 15us) server_negotiation.cc:407] Connection header received
0810 07:14:48.716039 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:48.716039 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:48.716059 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:48.716152 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:14:48.716299 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.716856 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.717942 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.718190 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.718239 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:48.718436 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:14:48.718541 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:14:48.718594 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:48.718663 (+ 69us) server_negotiation.cc:299] Negotiation successful
0810 07:14:48.718680 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75,"thread_start_us":52,"threads_started":1}
I20260810 07:14:48.724313 18028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:48.721742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:40813 (local address 127.15.217.66:41727)
0810 07:14:48.721785 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:48.721788 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:48.721998 (+ 210us) server_negotiation.cc:407] Connection header received
0810 07:14:48.722064 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:48.722065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:48.722148 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:48.722212 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:14:48.722320 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.722780 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.723568 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.723771 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.723847 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:48.724022 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:14:48.724059 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:14:48.724109 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:48.724208 (+ 99us) server_negotiation.cc:299] Negotiation successful
0810 07:14:48.724231 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 07:14:48.724372 18095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:48.721696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:41727 (local address 127.15.217.65:40813)
0810 07:14:48.721913 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:48.721925 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:48.722008 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:48.722217 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:48.722218 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:48.722220 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:14:48.722266 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:48.722268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.722860 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.722861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:48.723473 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:48.723475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.723845 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.723846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:48.723910 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:48.723926 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:14:48.724140 (+ 214us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:14:48.724142 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 07:14:48.724192 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 07:14:48.724238 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":85,"threads_started":1}
I20260810 07:14:48.727415 17955 raft_consensus.cc:1261] T 7b5aa8959ffb4445b997eb5387b704cb P b5be665e1ad04507887cec278a5efbfd [term 1 FOLLOWER]: Refusing update from remote peer 29fc149bdd0f49db9423c9ba867133ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:14:48.727757 17804 raft_consensus.cc:1261] T 7b5aa8959ffb4445b997eb5387b704cb P c87d93dff03f4a8dac67eb164d56274a [term 1 FOLLOWER]: Refusing update from remote peer 29fc149bdd0f49db9423c9ba867133ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:14:48.728070 18038 consensus_queue.cc:1035] T 7b5aa8959ffb4445b997eb5387b704cb P 29fc149bdd0f49db9423c9ba867133ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5be665e1ad04507887cec278a5efbfd" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:14:48.728339 18063 consensus_queue.cc:1035] T 7b5aa8959ffb4445b997eb5387b704cb P 29fc149bdd0f49db9423c9ba867133ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "c87d93dff03f4a8dac67eb164d56274a" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 41727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:14:48.738674 18091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:48.735475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:45289 (local address 127.15.217.65:34917)
0810 07:14:48.735517 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:48.735519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:48.735743 (+ 224us) server_negotiation.cc:407] Connection header received
0810 07:14:48.735792 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:48.735793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:48.735805 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:48.735860 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:14:48.735948 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.736688 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.737900 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.738185 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.738256 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:48.738398 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:14:48.738433 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:14:48.738485 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:48.738551 (+ 66us) server_negotiation.cc:299] Negotiation successful
0810 07:14:48.738576 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 07:14:48.738699 18100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:48.735413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:34917 (local address 127.15.217.66:45289)
0810 07:14:48.735663 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:48.735673 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:48.735729 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:48.735862 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:48.735863 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:48.735864 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:14:48.735909 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:48.735911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.736810 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.736811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:48.737823 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:48.737827 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.738260 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.738260 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:48.738301 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:48.738318 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:14:48.738492 (+ 174us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:14:48.738493 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 07:14:48.738528 (+ 35us) client_negotiation.cc:240] Negotiation successful
0810 07:14:48.738618 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":147,"threads_started":1}
I20260810 07:14:48.749014 17472 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:14:48.749148 17472 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:14:48.749866 17472 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 36a23f9d47b9445884702e5af00539f6: Sending DeleteTablet for 3 replicas of tablet 7b5aa8959ffb4445b997eb5387b704cb
I20260810 07:14:48.750267 17622 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b5aa8959ffb4445b997eb5387b704cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:14:48 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:49574
I20260810 07:14:48.750294 17933 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b5aa8959ffb4445b997eb5387b704cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:14:48 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:58176
I20260810 07:14:48.750650 18106 tablet_replica.cc:330] T 7b5aa8959ffb4445b997eb5387b704cb P b5be665e1ad04507887cec278a5efbfd: stopping tablet replica
I20260810 07:14:48.750722 18106 raft_consensus.cc:2229] T 7b5aa8959ffb4445b997eb5387b704cb P b5be665e1ad04507887cec278a5efbfd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:14:48.750761 18106 raft_consensus.cc:2258] T 7b5aa8959ffb4445b997eb5387b704cb P b5be665e1ad04507887cec278a5efbfd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:14:48.751073 18106 ts_tablet_manager.cc:1905] T 7b5aa8959ffb4445b997eb5387b704cb P b5be665e1ad04507887cec278a5efbfd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:14:48.751719 17784 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b5aa8959ffb4445b997eb5387b704cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:14:48 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:42398
I20260810 07:14:48.752314 18108 tablet_replica.cc:330] T 7b5aa8959ffb4445b997eb5387b704cb P c87d93dff03f4a8dac67eb164d56274a: stopping tablet replica
I20260810 07:14:48.752427 18108 raft_consensus.cc:2229] T 7b5aa8959ffb4445b997eb5387b704cb P c87d93dff03f4a8dac67eb164d56274a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:14:48.752836 18108 raft_consensus.cc:2258] T 7b5aa8959ffb4445b997eb5387b704cb P c87d93dff03f4a8dac67eb164d56274a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:14:48.753293 18108 ts_tablet_manager.cc:1905] T 7b5aa8959ffb4445b997eb5387b704cb P c87d93dff03f4a8dac67eb164d56274a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:14:48.753793 18106 ts_tablet_manager.cc:1918] T 7b5aa8959ffb4445b997eb5387b704cb P b5be665e1ad04507887cec278a5efbfd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:14:48.753870 18106 log.cc:1196] T 7b5aa8959ffb4445b997eb5387b704cb P b5be665e1ad04507887cec278a5efbfd: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-2/wal/wals/7b5aa8959ffb4445b997eb5387b704cb
I20260810 07:14:48.754173 18106 ts_tablet_manager.cc:1939] T 7b5aa8959ffb4445b997eb5387b704cb P b5be665e1ad04507887cec278a5efbfd: Deleting consensus metadata
I20260810 07:14:48.754547 17457 catalog_manager.cc:4841] TS b5be665e1ad04507887cec278a5efbfd (127.15.217.67:42441): tablet 7b5aa8959ffb4445b997eb5387b704cb (table test-table [id=dabec47624ab4cccb369f3051beb6eef]) successfully deleted
I20260810 07:14:48.755805 18108 ts_tablet_manager.cc:1918] T 7b5aa8959ffb4445b997eb5387b704cb P c87d93dff03f4a8dac67eb164d56274a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:14:48.755872 18108 log.cc:1196] T 7b5aa8959ffb4445b997eb5387b704cb P c87d93dff03f4a8dac67eb164d56274a: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-1/wal/wals/7b5aa8959ffb4445b997eb5387b704cb
I20260810 07:14:48.756131 18108 ts_tablet_manager.cc:1939] T 7b5aa8959ffb4445b997eb5387b704cb P c87d93dff03f4a8dac67eb164d56274a: Deleting consensus metadata
I20260810 07:14:48.756536 18104 tablet_replica.cc:330] T 7b5aa8959ffb4445b997eb5387b704cb P 29fc149bdd0f49db9423c9ba867133ce: stopping tablet replica
I20260810 07:14:48.756619 18104 raft_consensus.cc:2229] T 7b5aa8959ffb4445b997eb5387b704cb P 29fc149bdd0f49db9423c9ba867133ce [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:14:48.756603 17452 catalog_manager.cc:4841] TS c87d93dff03f4a8dac67eb164d56274a (127.15.217.66:41727): tablet 7b5aa8959ffb4445b997eb5387b704cb (table test-table [id=dabec47624ab4cccb369f3051beb6eef]) successfully deleted
I20260810 07:14:48.756721 18104 raft_consensus.cc:2258] T 7b5aa8959ffb4445b997eb5387b704cb P 29fc149bdd0f49db9423c9ba867133ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:14:48.757040 18104 ts_tablet_manager.cc:1905] T 7b5aa8959ffb4445b997eb5387b704cb P 29fc149bdd0f49db9423c9ba867133ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:14:48.759531 18104 ts_tablet_manager.cc:1918] T 7b5aa8959ffb4445b997eb5387b704cb P 29fc149bdd0f49db9423c9ba867133ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:14:48.759672 18104 log.cc:1196] T 7b5aa8959ffb4445b997eb5387b704cb P 29fc149bdd0f49db9423c9ba867133ce: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346081759552-16229-0/minicluster-data/ts-0/wal/wals/7b5aa8959ffb4445b997eb5387b704cb
I20260810 07:14:48.759923 18104 ts_tablet_manager.cc:1939] T 7b5aa8959ffb4445b997eb5387b704cb P 29fc149bdd0f49db9423c9ba867133ce: Deleting consensus metadata
I20260810 07:14:48.760799 17452 catalog_manager.cc:4841] TS 29fc149bdd0f49db9423c9ba867133ce (127.15.217.65:34917): tablet 7b5aa8959ffb4445b997eb5387b704cb (table test-table [id=dabec47624ab4cccb369f3051beb6eef]) successfully deleted
Aug 10 07:14:48 dist-test-slave-10pc krb5kdc[17418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346088, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.217.65@KRBTEST.COM
I20260810 07:14:48.777808 18091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:48.752164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49586 (local address 127.15.217.65:34917)
0810 07:14:48.752223 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:48.752226 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:48.764690 (+ 12464us) server_negotiation.cc:407] Connection header received
0810 07:14:48.764794 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:48.764795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:48.764811 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:48.764887 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:48.766080 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.766991 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.768167 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.768522 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.768600 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:48.773090 (+ 4490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:48.773101 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:48.773101 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:48.773111 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:48.775927 (+ 2816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:48.776380 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:48.776385 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:48.776385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:48.776636 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:48.776891 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:48.776892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:48.776892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:48.777230 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:48.777527 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:48.777582 (+ 55us) server_negotiation.cc:299] Negotiation successful
0810 07:14:48.777689 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260810 07:14:48.778111 17694 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49586
I20260810 07:14:48.788089 17525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:48.779328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52160 (local address 127.15.217.126:35039)
0810 07:14:48.779379 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:48.779381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:48.779391 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:14:48.779454 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:48.779455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:48.779467 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:48.779536 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:48.780260 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.781149 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.782235 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:48.782590 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:48.782680 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:48.784217 (+ 1537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:48.784219 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:48.784219 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:48.784228 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:48.786304 (+ 2076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:48.786810 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:48.786812 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:48.786813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:48.786887 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:48.787351 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:48.787352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:48.787353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:48.787527 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:48.787648 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:48.787860 (+ 212us) server_negotiation.cc:299] Negotiation successful
0810 07:14:48.787956 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260810 07:14:48.788331 17472 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52160
I20260810 07:14:48.789276 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 17550
I20260810 07:14:48.798384 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 17709
I20260810 07:14:48.807478 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 17863
I20260810 07:14:48.815083 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 17433
2026-08-10T07:14:48Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2711 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:14:48 dist-test-slave-10pc krb5kdc[18115](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:14:48 dist-test-slave-10pc krb5kdc[18115](info): set up 2 sockets
Aug 10 07:14:48 dist-test-slave-10pc krb5kdc[18115](info): commencing operation
krb5kdc: starting...
W20260810 07:14:49.906950 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:14:49 dist-test-slave-10pc krb5kdc[18115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:14:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:14:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
I20260810 07:14:50.041432 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:41785
--webserver_interface=127.15.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:39207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:41785
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:14:50.127657 18129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:14:50.127907 18129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:14:50.127946 18129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:14:50.129177 18129 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:14:50.129240 18129 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 07:14:50.129261 18129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:14:50.129285 18129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:14:50.129325 18129 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:14:50.129357 18129 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:14:50.130858 18129 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:39207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:41785
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:41785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:14:50.131143 18129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:14:50.131377 18129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:14:50.136534 18136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:50.136557 18138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:50.136540 18135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:14:50.136658 18129 server_base.cc:1034] running on GCE node
I20260810 07:14:50.137063 18129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:14:50.137319 18129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:14:50.138463 18129 hybrid_clock.cc:648] HybridClock initialized: now 1786346090138451 us; error 29 us; skew 500 ppm
Aug 10 07:14:50 dist-test-slave-10pc krb5kdc[18115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346090, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:14:50.141292 18129 init.cc:377] Logged in from keytab as kudu/127.15.217.126@KRBTEST.COM (short username kudu)
I20260810 07:14:50.141768 18129 webserver.cc:460] Webserver started at http://127.15.217.126:41103/ using document root <none> and password file <none>
I20260810 07:14:50.142014 18129 fs_manager.cc:359] Metadata directory not provided
I20260810 07:14:50.142083 18129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:14:50.142244 18129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:14:50.143399 18129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "1b66515d7fd64e6dbd7495ec1558830f"
format_stamp: "Formatted at 2026-08-10 07:14:50 on dist-test-slave-10pc"
server_key: "b1a4c9f22afa0ad694da75e83b9cbe93"
server_key_iv: "ea89c8428db56738af688e917ff90d58"
server_key_version: "encryptionkey@0"
I20260810 07:14:50.143816 18129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "1b66515d7fd64e6dbd7495ec1558830f"
format_stamp: "Formatted at 2026-08-10 07:14:50 on dist-test-slave-10pc"
server_key: "b1a4c9f22afa0ad694da75e83b9cbe93"
server_key_iv: "ea89c8428db56738af688e917ff90d58"
server_key_version: "encryptionkey@0"
I20260810 07:14:50.145403 18129 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:14:50.146238 18146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:50.146432 18129 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:14:50.146540 18129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "1b66515d7fd64e6dbd7495ec1558830f"
format_stamp: "Formatted at 2026-08-10 07:14:50 on dist-test-slave-10pc"
server_key: "b1a4c9f22afa0ad694da75e83b9cbe93"
server_key_iv: "ea89c8428db56738af688e917ff90d58"
server_key_version: "encryptionkey@0"
I20260810 07:14:50.146618 18129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:14:50.172658 18129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:14:50.175276 18129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:14:50.175468 18129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:14:50.180369 18129 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:41785
I20260810 07:14:50.180509 18212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:41785 every 8 connection(s)
I20260810 07:14:50.180841 18129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:14:50.181583 18213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:14:50.184067 18213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b66515d7fd64e6dbd7495ec1558830f: Bootstrap starting.
I20260810 07:14:50.184743 18213 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b66515d7fd64e6dbd7495ec1558830f: Neither blocks nor log segments found. Creating new log.
I20260810 07:14:50.185087 18213 log.cc:824] T 00000000000000000000000000000000 P 1b66515d7fd64e6dbd7495ec1558830f: Log is configured to *not* fsync() on all Append() calls
I20260810 07:14:50.185860 18213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b66515d7fd64e6dbd7495ec1558830f: No bootstrap required, opened a new log
I20260810 07:14:50.187655 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 18129
I20260810 07:14:50.187769 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:14:50.188043 16229 external_mini_cluster.cc:1428] Setting key 9b8ee3d800d020fcbef05fc211b694b9
I20260810 07:14:50.188405 18213 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b66515d7fd64e6dbd7495ec1558830f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b66515d7fd64e6dbd7495ec1558830f" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41785 } }
I20260810 07:14:50.188664 18213 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b66515d7fd64e6dbd7495ec1558830f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:14:50.188758 18213 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b66515d7fd64e6dbd7495ec1558830f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b66515d7fd64e6dbd7495ec1558830f, State: Initialized, Role: FOLLOWER
I20260810 07:14:50.188942 18213 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b66515d7fd64e6dbd7495ec1558830f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b66515d7fd64e6dbd7495ec1558830f" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41785 } }
I20260810 07:14:50.189019 18213 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b66515d7fd64e6dbd7495ec1558830f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:14:50.189041 18213 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b66515d7fd64e6dbd7495ec1558830f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:14:50.189103 18213 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b66515d7fd64e6dbd7495ec1558830f [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:14:50.189924 18213 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b66515d7fd64e6dbd7495ec1558830f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b66515d7fd64e6dbd7495ec1558830f" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41785 } }
I20260810 07:14:50.190050 18213 leader_election.cc:302] T 00000000000000000000000000000000 P 1b66515d7fd64e6dbd7495ec1558830f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b66515d7fd64e6dbd7495ec1558830f; no voters:
I20260810 07:14:50.190236 18213 leader_election.cc:288] T 00000000000000000000000000000000 P 1b66515d7fd64e6dbd7495ec1558830f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:14:50.190268 18219 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b66515d7fd64e6dbd7495ec1558830f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:14:50.190461 18219 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b66515d7fd64e6dbd7495ec1558830f [term 1 LEADER]: Becoming Leader. State: Replica: 1b66515d7fd64e6dbd7495ec1558830f, State: Running, Role: LEADER
I20260810 07:14:50.190516 18213 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b66515d7fd64e6dbd7495ec1558830f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:14:50.190596 18219 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b66515d7fd64e6dbd7495ec1558830f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b66515d7fd64e6dbd7495ec1558830f" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41785 } }
I20260810 07:14:50.191051 18219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b66515d7fd64e6dbd7495ec1558830f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b66515d7fd64e6dbd7495ec1558830f. Latest consensus state: current_term: 1 leader_uuid: "1b66515d7fd64e6dbd7495ec1558830f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b66515d7fd64e6dbd7495ec1558830f" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41785 } } }
I20260810 07:14:50.191061 18220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b66515d7fd64e6dbd7495ec1558830f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b66515d7fd64e6dbd7495ec1558830f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b66515d7fd64e6dbd7495ec1558830f" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41785 } } }
I20260810 07:14:50.191238 18219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b66515d7fd64e6dbd7495ec1558830f [sys.catalog]: This master's current role is: LEADER
I20260810 07:14:50.191244 18220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b66515d7fd64e6dbd7495ec1558830f [sys.catalog]: This master's current role is: LEADER
I20260810 07:14:50.191627 18229 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:14:50.192505 18229 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 07:14:50 dist-test-slave-10pc krb5kdc[18115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:14:50.196386 18229 catalog_manager.cc:1347] Generated new cluster ID: b3b865110e3e461996c1f487f8ae02e3
I20260810 07:14:50.196476 18229 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:14:50.200647 18218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:50.188631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58910 (local address 127.15.217.126:41785)
0810 07:14:50.188868 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:50.188880 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:50.188916 (+ 36us) server_negotiation.cc:407] Connection header received
0810 07:14:50.189075 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:50.189083 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:50.189144 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:50.189281 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:50.190152 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.191695 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.192925 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.193566 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.193679 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:50.196379 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:50.196383 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:50.196383 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:50.196396 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:50.198893 (+ 2497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:50.199299 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:50.199303 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:50.199303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:50.199427 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:50.199655 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:50.199656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:50.199656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:50.199930 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:50.200095 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:50.200209 (+ 114us) server_negotiation.cc:299] Negotiation successful
0810 07:14:50.200380 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":122,"threads_started":1}
I20260810 07:14:50.208637 18229 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:14:50.209295 18229 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:14:50.215865 18229 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b66515d7fd64e6dbd7495ec1558830f: Generated new TSK 0
I20260810 07:14:50.216097 18229 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab.
Entry for principal kudu/127.15.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab.
I20260810 07:14:50.254876 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.65:0
--local_ip_for_outbound_sockets=127.15.217.65
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:41785
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:39207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:14:50.365043 18242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:14:50.365267 18242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:14:50.365295 18242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:14:50.366534 18242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:14:50.366585 18242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:14:50.366627 18242 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 07:14:50.366655 18242 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 07:14:50.366683 18242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.65
I20260810 07:14:50.368248 18242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:39207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.217.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.217.126:41785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.65
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:14:50.368575 18242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:14:50.368844 18242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:14:50.374377 18248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:50.374377 18249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:14:50.374410 18242 server_base.cc:1034] running on GCE node
W20260810 07:14:50.374382 18251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:14:50.374903 18242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:14:50.375187 18242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:14:50.376332 18242 hybrid_clock.cc:648] HybridClock initialized: now 1786346090376321 us; error 35 us; skew 500 ppm
Aug 10 07:14:50 dist-test-slave-10pc krb5kdc[18115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346090, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:14:50.379076 18242 init.cc:377] Logged in from keytab as kudu/127.15.217.65@KRBTEST.COM (short username kudu)
I20260810 07:14:50.379575 18242 webserver.cc:460] Webserver started at http://127.15.217.65:38905/ using document root <none> and password file <none>
I20260810 07:14:50.379829 18242 fs_manager.cc:359] Metadata directory not provided
I20260810 07:14:50.379878 18242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:14:50.380026 18242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:14:50.381230 18242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-0/data/instance:
uuid: "404270aa756047a288291fd808377592"
format_stamp: "Formatted at 2026-08-10 07:14:50 on dist-test-slave-10pc"
server_key: "7292991c6e3567941ad8fb6b7aef8f80"
server_key_iv: "51f14ca2cb39c5eb5f93b0ccf86dfbd0"
server_key_version: "encryptionkey@0"
I20260810 07:14:50.381626 18242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance:
uuid: "404270aa756047a288291fd808377592"
format_stamp: "Formatted at 2026-08-10 07:14:50 on dist-test-slave-10pc"
server_key: "7292991c6e3567941ad8fb6b7aef8f80"
server_key_iv: "51f14ca2cb39c5eb5f93b0ccf86dfbd0"
server_key_version: "encryptionkey@0"
I20260810 07:14:50.383015 18242 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 07:14:50.383823 18260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:50.384016 18242 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:14:50.384121 18242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-0/wal
uuid: "404270aa756047a288291fd808377592"
format_stamp: "Formatted at 2026-08-10 07:14:50 on dist-test-slave-10pc"
server_key: "7292991c6e3567941ad8fb6b7aef8f80"
server_key_iv: "51f14ca2cb39c5eb5f93b0ccf86dfbd0"
server_key_version: "encryptionkey@0"
I20260810 07:14:50.384205 18242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:14:50.392954 18242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:14:50.395303 18242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:14:50.395498 18242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:14:50.396099 18242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:14:50.396169 18242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:50.396245 18242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:14:50.396281 18242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:50.404364 18242 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.65:34161
I20260810 07:14:50.404536 18387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.65:34161 every 8 connection(s)
Aug 10 07:14:50 dist-test-slave-10pc krb5kdc[18115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346090, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:14:50.404911 18242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
I20260810 07:14:50.408252 18218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:50.396719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:44753 (local address 127.15.217.126:41785)
0810 07:14:50.396770 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:50.396772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:50.397038 (+ 266us) server_negotiation.cc:407] Connection header received
0810 07:14:50.397555 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:50.397558 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:50.397576 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:50.397661 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:50.398520 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.399463 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.401405 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.401844 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.401932 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:50.405420 (+ 3488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:50.405431 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:50.405431 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:50.405441 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:50.406947 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:50.407387 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:50.407388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:50.407388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:50.407454 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:50.407694 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:50.407695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:50.407696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:50.407816 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:50.407874 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:50.408107 (+ 233us) server_negotiation.cc:299] Negotiation successful
0810 07:14:50.408184 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 07:14:50.408414 18270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:50.396735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41785 (local address 127.15.217.65:44753)
0810 07:14:50.396909 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:50.396956 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:50.397450 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:50.397715 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:50.397718 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:50.398083 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:14:50.398395 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:50.398404 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.399615 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.399617 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:50.401290 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:50.401295 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.401976 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.401977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:50.402148 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:50.402785 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:14:50.402799 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:14:50.405263 (+ 2464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:14:50.407040 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:50.407052 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:50.407054 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:50.407293 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:50.407526 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:50.407527 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:50.407527 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:50.407630 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:50.407873 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:14:50.407878 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:14:50.408003 (+ 125us) client_negotiation.cc:769] Sending connection context
0810 07:14:50.408084 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 07:14:50.408215 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":105,"threads_started":1}
I20260810 07:14:50.409017 18388 heartbeater.cc:347] Connected to a master server at 127.15.217.126:41785
I20260810 07:14:50.409124 18388 heartbeater.cc:464] Registering TS with master...
I20260810 07:14:50.409317 18388 heartbeater.cc:511] Master 127.15.217.126:41785 requested a full tablet report, sending...
I20260810 07:14:50.409852 18162 ts_manager.cc:194] Registered new tserver with Master: 404270aa756047a288291fd808377592 (127.15.217.65:34161)
I20260810 07:14:50.410804 18162 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.65@KRBTEST.COM'} at 127.15.217.65:44753
I20260810 07:14:50.411770 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 18242
I20260810 07:14:50.411873 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance
I20260810 07:14:50.412091 16229 external_mini_cluster.cc:1428] Setting key 58b8b336441f4dbe30f2d14150c5a5aa
I20260810 07:14:50.416661 18218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:50.410235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:60641 (local address 127.15.217.126:41785)
0810 07:14:50.410279 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:50.410281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:50.410354 (+ 73us) server_negotiation.cc:407] Connection header received
0810 07:14:50.410420 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:50.410421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:50.410429 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:50.410470 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:50.410869 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.411420 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.412267 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.412571 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.412628 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:50.413592 (+ 964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:50.413594 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:50.413594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:50.413602 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:50.415207 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:50.415719 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:50.415720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:50.415720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:50.415790 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:50.416036 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:50.416037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:50.416038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:50.416147 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:50.416207 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:50.416419 (+ 212us) server_negotiation.cc:299] Negotiation successful
0810 07:14:50.416591 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 07:14:50.416698 18395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:50.410148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41785 (local address 127.15.217.65:60641)
0810 07:14:50.410292 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:50.410303 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:50.410364 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:50.410483 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:50.410484 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:50.410720 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:14:50.410789 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:50.410791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.411522 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.411523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:50.412144 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:50.412147 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.412641 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.412643 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:50.412732 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:50.413220 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:14:50.413225 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:14:50.413502 (+ 277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:14:50.415305 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:50.415306 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:50.415307 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:50.415613 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:50.415871 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:50.415872 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:50.415872 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:50.415959 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:50.416219 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:14:50.416220 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:14:50.416281 (+ 61us) client_negotiation.cc:769] Sending connection context
0810 07:14:50.416368 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 07:14:50.416536 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":80,"threads_started":1}
WARNING: no policy specified for kudu/127.15.217.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab.
Entry for principal kudu/127.15.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab.
I20260810 07:14:50.442842 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.66:0
--local_ip_for_outbound_sockets=127.15.217.66
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:41785
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:39207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:14:50.528753 18398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:14:50.528962 18398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:14:50.528988 18398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:14:50.530094 18398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:14:50.530131 18398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:14:50.530151 18398 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 07:14:50.530170 18398 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 07:14:50.530187 18398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.66
I20260810 07:14:50.531622 18398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:39207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.217.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.217.126:41785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.66
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:14:50.531821 18398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:14:50.532004 18398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:14:50.537112 18406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:50.537142 18407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:50.537142 18409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:14:50.537333 18398 server_base.cc:1034] running on GCE node
I20260810 07:14:50.537575 18398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:14:50.537833 18398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:14:50.539000 18398 hybrid_clock.cc:648] HybridClock initialized: now 1786346090538987 us; error 31 us; skew 500 ppm
Aug 10 07:14:50 dist-test-slave-10pc krb5kdc[18115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346090, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:14:50.541996 18398 init.cc:377] Logged in from keytab as kudu/127.15.217.66@KRBTEST.COM (short username kudu)
I20260810 07:14:50.542508 18398 webserver.cc:460] Webserver started at http://127.15.217.66:34559/ using document root <none> and password file <none>
I20260810 07:14:50.542775 18398 fs_manager.cc:359] Metadata directory not provided
I20260810 07:14:50.542842 18398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:14:50.543000 18398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:14:50.544100 18398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-1/data/instance:
uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3"
format_stamp: "Formatted at 2026-08-10 07:14:50 on dist-test-slave-10pc"
server_key: "d821d9631c78a400a5f0110d617e32f9"
server_key_iv: "a8599261366a8bb885871bf62327b692"
server_key_version: "encryptionkey@0"
I20260810 07:14:50.544565 18398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance:
uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3"
format_stamp: "Formatted at 2026-08-10 07:14:50 on dist-test-slave-10pc"
server_key: "d821d9631c78a400a5f0110d617e32f9"
server_key_iv: "a8599261366a8bb885871bf62327b692"
server_key_version: "encryptionkey@0"
I20260810 07:14:50.546005 18398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:14:50.546828 18416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:50.547010 18398 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:14:50.547113 18398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-1/wal
uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3"
format_stamp: "Formatted at 2026-08-10 07:14:50 on dist-test-slave-10pc"
server_key: "d821d9631c78a400a5f0110d617e32f9"
server_key_iv: "a8599261366a8bb885871bf62327b692"
server_key_version: "encryptionkey@0"
I20260810 07:14:50.547207 18398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:14:50.566874 18398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:14:50.569060 18398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:14:50.569236 18398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:14:50.569869 18398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:14:50.569929 18398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:50.569962 18398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:14:50.569991 18398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 07:14:50 dist-test-slave-10pc krb5kdc[18115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346090, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:14:50.577839 18398 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.66:33449
I20260810 07:14:50.577937 18538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.66:33449 every 8 connection(s)
I20260810 07:14:50.578291 18398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
I20260810 07:14:50.579085 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 18398
I20260810 07:14:50.579198 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance
I20260810 07:14:50.579416 16229 external_mini_cluster.cc:1428] Setting key f20bf34936528e2a8fda3b274b5418d3
I20260810 07:14:50.581507 18218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:50.570351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:46933 (local address 127.15.217.126:41785)
0810 07:14:50.570405 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:50.570407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:50.570663 (+ 256us) server_negotiation.cc:407] Connection header received
0810 07:14:50.571283 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:50.571285 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:50.571300 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:50.571373 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:50.572241 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.572881 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.574042 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.574305 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.574385 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:50.578081 (+ 3696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:50.578083 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:50.578084 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:50.578093 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:50.579934 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:50.580413 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:50.580414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:50.580414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:50.580507 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:50.580758 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:50.580759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:50.580759 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:50.580918 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:50.580997 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:50.581269 (+ 272us) server_negotiation.cc:299] Negotiation successful
0810 07:14:50.581368 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 07:14:50.581573 18426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:50.570329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41785 (local address 127.15.217.66:46933)
0810 07:14:50.570510 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:50.570567 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:50.571162 (+ 595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:50.571459 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:50.571463 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:50.571814 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:14:50.572137 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:50.572143 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.572993 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.572994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:50.573919 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:50.573922 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.574395 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.574396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:50.574608 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:50.575224 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:14:50.575237 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:14:50.577954 (+ 2717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:14:50.580020 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:50.580027 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:50.580029 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:50.580336 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:50.580583 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:50.580584 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:50.580584 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:50.580696 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:50.580994 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:14:50.581000 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:14:50.581132 (+ 132us) client_negotiation.cc:769] Sending connection context
0810 07:14:50.581225 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 07:14:50.581368 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":91,"threads_started":1}
I20260810 07:14:50.583315 18539 heartbeater.cc:347] Connected to a master server at 127.15.217.126:41785
I20260810 07:14:50.583590 18539 heartbeater.cc:464] Registering TS with master...
I20260810 07:14:50.584703 18539 heartbeater.cc:511] Master 127.15.217.126:41785 requested a full tablet report, sending...
I20260810 07:14:50.585146 18163 ts_manager.cc:194] Registered new tserver with Master: f3e90ba48a6a44b5bb4dfbc06888f1c3 (127.15.217.66:33449)
I20260810 07:14:50.585546 18163 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.66@KRBTEST.COM'} at 127.15.217.66:46933
I20260810 07:14:50.590421 18218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:50.583478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:46179 (local address 127.15.217.126:41785)
0810 07:14:50.583526 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:50.583529 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:50.583635 (+ 106us) server_negotiation.cc:407] Connection header received
0810 07:14:50.583706 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:50.583707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:50.583718 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:50.583774 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:50.584096 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.584697 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.586077 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.586347 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.586427 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:50.587624 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:50.587625 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:50.587625 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:50.587632 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:50.589105 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:50.589542 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:50.589543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:50.589543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:50.589592 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:50.589803 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:50.589803 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:50.589803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:50.589941 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:50.590029 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:50.590244 (+ 215us) server_negotiation.cc:299] Negotiation successful
0810 07:14:50.590331 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
WARNING: no policy specified for kudu/127.15.217.67@KRBTEST.COM; defaulting to no policy
I20260810 07:14:50.590462 18548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:50.583409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41785 (local address 127.15.217.66:46179)
0810 07:14:50.583558 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:50.583569 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:50.583649 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:50.583779 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:50.583780 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:50.583967 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:14:50.584021 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:50.584023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.584866 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.584867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:50.585987 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:50.585990 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.586439 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.586440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:50.586549 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:50.586991 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:14:50.586997 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:14:50.587530 (+ 533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:14:50.589191 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:50.589192 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:50.589192 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:50.589458 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:50.589657 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:50.589659 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:50.589659 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:50.589732 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:50.590033 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:14:50.590034 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:14:50.590114 (+ 80us) client_negotiation.cc:769] Sending connection context
0810 07:14:50.590202 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 07:14:50.590314 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":80,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab.
Entry for principal kudu/127.15.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab.
I20260810 07:14:50.611879 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.67:0
--local_ip_for_outbound_sockets=127.15.217.67
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:41785
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:39207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:14:50.699172 18551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:14:50.699386 18551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:14:50.699414 18551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:14:50.700644 18551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:14:50.700686 18551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:14:50.700708 18551 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 07:14:50.700721 18551 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 07:14:50.700773 18551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.67
I20260810 07:14:50.702240 18551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:39207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.217.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.217.126:41785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.67
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:14:50.702494 18551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:14:50.702739 18551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:14:50.707957 18558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:50.707959 18560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:50.707975 18556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:14:50.708251 18551 server_base.cc:1034] running on GCE node
I20260810 07:14:50.708505 18551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:14:50.708768 18551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:14:50.709930 18551 hybrid_clock.cc:648] HybridClock initialized: now 1786346090709911 us; error 38 us; skew 500 ppm
Aug 10 07:14:50 dist-test-slave-10pc krb5kdc[18115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346090, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:14:50.712677 18551 init.cc:377] Logged in from keytab as kudu/127.15.217.67@KRBTEST.COM (short username kudu)
I20260810 07:14:50.713125 18551 webserver.cc:460] Webserver started at http://127.15.217.67:32797/ using document root <none> and password file <none>
I20260810 07:14:50.713384 18551 fs_manager.cc:359] Metadata directory not provided
I20260810 07:14:50.713454 18551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:14:50.713594 18551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:14:50.714699 18551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-2/data/instance:
uuid: "cd1618f314b441a99f3374fa93174aa1"
format_stamp: "Formatted at 2026-08-10 07:14:50 on dist-test-slave-10pc"
server_key: "d73760486a7e9397910bdd072820dafc"
server_key_iv: "24d45e99fceb5705d0883c10cf8e397f"
server_key_version: "encryptionkey@0"
I20260810 07:14:50.715080 18551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance:
uuid: "cd1618f314b441a99f3374fa93174aa1"
format_stamp: "Formatted at 2026-08-10 07:14:50 on dist-test-slave-10pc"
server_key: "d73760486a7e9397910bdd072820dafc"
server_key_iv: "24d45e99fceb5705d0883c10cf8e397f"
server_key_version: "encryptionkey@0"
I20260810 07:14:50.716704 18551 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:14:50.717458 18569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:50.717638 18551 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 07:14:50.717747 18551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-2/wal
uuid: "cd1618f314b441a99f3374fa93174aa1"
format_stamp: "Formatted at 2026-08-10 07:14:50 on dist-test-slave-10pc"
server_key: "d73760486a7e9397910bdd072820dafc"
server_key_iv: "24d45e99fceb5705d0883c10cf8e397f"
server_key_version: "encryptionkey@0"
I20260810 07:14:50.717824 18551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:14:50.729027 18551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:14:50.731470 18551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:14:50.731642 18551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:14:50.732189 18551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:14:50.732254 18551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:50.732309 18551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:14:50.732353 18551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:50.739768 18551 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.67:34305
I20260810 07:14:50.739877 18691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.67:34305 every 8 connection(s)
Aug 10 07:14:50 dist-test-slave-10pc krb5kdc[18115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346090, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:14:50.740214 18551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
I20260810 07:14:50.743288 18218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:50.732818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:54445 (local address 127.15.217.126:41785)
0810 07:14:50.732886 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:50.732890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:50.733105 (+ 215us) server_negotiation.cc:407] Connection header received
0810 07:14:50.733754 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:50.733756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:50.733771 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:50.733860 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:50.734758 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.735612 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.737065 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.737433 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.737523 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:50.740566 (+ 3043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:50.740568 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:50.740569 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:50.740579 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:50.741842 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:50.742246 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:50.742247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:50.742248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:50.742316 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:50.742590 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:50.742591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:50.742592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:50.742725 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:50.742806 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:50.743103 (+ 297us) server_negotiation.cc:299] Negotiation successful
0810 07:14:50.743197 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 07:14:50.743471 18578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:50.732790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41785 (local address 127.15.217.67:54445)
0810 07:14:50.732971 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:50.733014 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:50.733640 (+ 626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:50.733949 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:50.733953 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:50.734322 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:14:50.734641 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:50.734649 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.735728 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.735729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:50.736946 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:50.736949 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.737557 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.737558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:50.737717 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:50.738391 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:14:50.738401 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:14:50.740431 (+ 2030us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:14:50.741915 (+ 1484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:50.741924 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:50.741926 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:50.742163 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:50.742385 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:50.742386 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:50.742387 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:50.742524 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:50.742802 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:14:50.742811 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:14:50.742955 (+ 144us) client_negotiation.cc:769] Sending connection context
0810 07:14:50.743065 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 07:14:50.743233 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":99,"threads_started":1}
I20260810 07:14:50.743989 18693 heartbeater.cc:347] Connected to a master server at 127.15.217.126:41785
I20260810 07:14:50.744102 18693 heartbeater.cc:464] Registering TS with master...
I20260810 07:14:50.744302 18693 heartbeater.cc:511] Master 127.15.217.126:41785 requested a full tablet report, sending...
I20260810 07:14:50.744741 18162 ts_manager.cc:194] Registered new tserver with Master: cd1618f314b441a99f3374fa93174aa1 (127.15.217.67:34305)
I20260810 07:14:50.745307 18162 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.67@KRBTEST.COM'} at 127.15.217.67:54445
I20260810 07:14:50.748310 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 18551
I20260810 07:14:50.748420 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance
I20260810 07:14:50.748651 16229 external_mini_cluster.cc:1428] Setting key fd1d4a624054b9bdbb21f72d020af0d6
I20260810 07:14:50.749680 16229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:14:50.751204 18700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:50.745098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41785 (local address 127.15.217.67:57627)
0810 07:14:50.745277 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:50.745286 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:50.745355 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:50.745502 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:50.745503 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:50.745678 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:14:50.745739 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:50.745741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.746385 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.746386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:50.746895 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:50.746897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.747258 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.747259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:50.747334 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:50.747720 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:14:50.747724 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:14:50.748042 (+ 318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:14:50.750066 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:50.750067 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:50.750068 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:50.750322 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:50.750557 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:50.750558 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:50.750558 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:50.750616 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:50.750935 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:14:50.750936 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:14:50.750977 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 07:14:50.751030 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 07:14:50.751116 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":2,"thread_start_us":85,"threads_started":1}
I20260810 07:14:50.751325 18218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:50.745172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:57627 (local address 127.15.217.126:41785)
0810 07:14:50.745213 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:50.745215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:50.745340 (+ 125us) server_negotiation.cc:407] Connection header received
0810 07:14:50.745413 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:50.745414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:50.745425 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:50.745496 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:50.745816 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.746294 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.746975 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.747186 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.747260 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:50.748158 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:50.748159 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:50.748160 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:50.748166 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:50.749978 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:50.750412 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:50.750413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:50.750414 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:50.750484 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:50.750723 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:50.750724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:50.750724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:50.750870 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:50.750941 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:50.751055 (+ 114us) server_negotiation.cc:299] Negotiation successful
0810 07:14:50.751138 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Aug 10 07:14:50 dist-test-slave-10pc krb5kdc[18115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 07:14:50 dist-test-slave-10pc krb5kdc[18115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:14:50.774518 18218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:50.767456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58920 (local address 127.15.217.126:41785)
0810 07:14:50.767534 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:50.767536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:50.767546 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:14:50.767574 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:50.767575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:50.767585 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:50.767648 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:50.768236 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.768820 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.769553 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.769733 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.769787 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:50.772005 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:50.772007 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:50.772007 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:50.772014 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:50.773477 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:50.773831 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:50.773832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:50.773833 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:50.773894 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:50.774073 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:50.774074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:50.774074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:50.774187 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:50.774264 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:50.774372 (+ 108us) server_negotiation.cc:299] Negotiation successful
0810 07:14:50.774438 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 07:14:50.776238 18162 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58920:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:14:50.777962 18162 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:14:50.784588 18713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:50.780385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37220 (local address 127.15.217.67:34305)
0810 07:14:50.780603 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:50.780609 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:50.780746 (+ 137us) server_negotiation.cc:407] Connection header received
0810 07:14:50.780835 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:50.780838 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:50.780856 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:50.781154 (+ 298us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:14:50.781300 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.782086 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.783518 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.784128 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.784218 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:50.784300 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:50.784427 (+ 127us) server_negotiation.cc:299] Negotiation successful
0810 07:14:50.784492 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":108,"threads_started":1}
I20260810 07:14:50.784582 18715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:50.780287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:34305 (local address 127.0.0.1:37220)
0810 07:14:50.780671 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:50.780684 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:50.780759 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:50.780984 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:50.780988 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:50.780990 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:14:50.781080 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:50.781082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.782224 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.782226 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:50.783427 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:50.783430 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.784235 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.784236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:50.784320 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:50.784334 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 07:14:50.784405 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 07:14:50.784486 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"mutex_wait_us":51,"thread_start_us":96,"threads_started":1}
I20260810 07:14:50.784754 18712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:50.780513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:33449 (local address 127.0.0.1:58216)
0810 07:14:50.780736 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:50.780745 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:50.780786 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:50.781137 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:50.781138 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:50.781139 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:14:50.781188 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:50.781190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.782511 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.782512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:50.783535 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:50.783537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.784498 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.784500 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:50.784575 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:50.784580 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:14:50.784612 (+ 32us) client_negotiation.cc:240] Negotiation successful
0810 07:14:50.784653 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":2,"thread_start_us":71,"threads_started":1}
I20260810 07:14:50.785223 18716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:50.780755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58216 (local address 127.15.217.66:33449)
0810 07:14:50.780921 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:50.780925 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:50.780933 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:14:50.781023 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:50.781026 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:50.781042 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:50.781263 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:14:50.781326 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.782427 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.783615 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.784362 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.784952 (+ 590us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:50.785036 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:50.785070 (+ 34us) server_negotiation.cc:299] Negotiation successful
0810 07:14:50.785111 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":94,"threads_started":1}
I20260810 07:14:50.785701 18619 tablet_service.cc:1467] Processing CreateTablet for tablet 428f04b9738c447fb48a08c0a6c00eca (DEFAULT_TABLE table=test-table [id=df6803d3aa9b44f69b769d3bfa31f4af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:14:50.785988 18711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:50.780280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:34161 (local address 127.0.0.1:34406)
0810 07:14:50.780482 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:50.780528 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:50.780601 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:50.781125 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:50.781126 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:50.781128 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:14:50.781188 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:50.781191 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.783543 (+ 2352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.783543 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:50.784663 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:50.784665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.785745 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.785746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:50.785805 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:50.785808 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:14:50.785878 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 07:14:50.785920 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":6,"thread_start_us":67,"threads_started":1}
I20260810 07:14:50.785897 18468 tablet_service.cc:1467] Processing CreateTablet for tablet 428f04b9738c447fb48a08c0a6c00eca (DEFAULT_TABLE table=test-table [id=df6803d3aa9b44f69b769d3bfa31f4af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:14:50.786049 18714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:50.780400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34406 (local address 127.15.217.65:34161)
0810 07:14:50.780894 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:50.780899 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:50.780909 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:14:50.780953 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:50.780955 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:50.780971 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:50.781050 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:14:50.781268 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.782427 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.784755 (+ 2328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.785580 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.785647 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:50.785706 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:50.785896 (+ 190us) server_negotiation.cc:299] Negotiation successful
0810 07:14:50.785938 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":162,"threads_started":1}
I20260810 07:14:50.786126 18619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 428f04b9738c447fb48a08c0a6c00eca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:14:50.786228 18468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 428f04b9738c447fb48a08c0a6c00eca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:14:50.786644 18317 tablet_service.cc:1467] Processing CreateTablet for tablet 428f04b9738c447fb48a08c0a6c00eca (DEFAULT_TABLE table=test-table [id=df6803d3aa9b44f69b769d3bfa31f4af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:14:50.786989 18317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 428f04b9738c447fb48a08c0a6c00eca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:14:50.789835 18717 tablet_bootstrap.cc:492] T 428f04b9738c447fb48a08c0a6c00eca P cd1618f314b441a99f3374fa93174aa1: Bootstrap starting.
I20260810 07:14:50.790218 18719 tablet_bootstrap.cc:492] T 428f04b9738c447fb48a08c0a6c00eca P 404270aa756047a288291fd808377592: Bootstrap starting.
I20260810 07:14:50.790604 18718 tablet_bootstrap.cc:492] T 428f04b9738c447fb48a08c0a6c00eca P f3e90ba48a6a44b5bb4dfbc06888f1c3: Bootstrap starting.
I20260810 07:14:50.790954 18717 tablet_bootstrap.cc:654] T 428f04b9738c447fb48a08c0a6c00eca P cd1618f314b441a99f3374fa93174aa1: Neither blocks nor log segments found. Creating new log.
I20260810 07:14:50.791098 18719 tablet_bootstrap.cc:654] T 428f04b9738c447fb48a08c0a6c00eca P 404270aa756047a288291fd808377592: Neither blocks nor log segments found. Creating new log.
I20260810 07:14:50.791313 18718 tablet_bootstrap.cc:654] T 428f04b9738c447fb48a08c0a6c00eca P f3e90ba48a6a44b5bb4dfbc06888f1c3: Neither blocks nor log segments found. Creating new log.
I20260810 07:14:50.791338 18717 log.cc:824] T 428f04b9738c447fb48a08c0a6c00eca P cd1618f314b441a99f3374fa93174aa1: Log is configured to *not* fsync() on all Append() calls
I20260810 07:14:50.791530 18719 log.cc:824] T 428f04b9738c447fb48a08c0a6c00eca P 404270aa756047a288291fd808377592: Log is configured to *not* fsync() on all Append() calls
I20260810 07:14:50.791808 18718 log.cc:824] T 428f04b9738c447fb48a08c0a6c00eca P f3e90ba48a6a44b5bb4dfbc06888f1c3: Log is configured to *not* fsync() on all Append() calls
I20260810 07:14:50.792280 18717 tablet_bootstrap.cc:492] T 428f04b9738c447fb48a08c0a6c00eca P cd1618f314b441a99f3374fa93174aa1: No bootstrap required, opened a new log
I20260810 07:14:50.792364 18717 ts_tablet_manager.cc:1397] T 428f04b9738c447fb48a08c0a6c00eca P cd1618f314b441a99f3374fa93174aa1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:14:50.792444 18719 tablet_bootstrap.cc:492] T 428f04b9738c447fb48a08c0a6c00eca P 404270aa756047a288291fd808377592: No bootstrap required, opened a new log
I20260810 07:14:50.792536 18719 ts_tablet_manager.cc:1397] T 428f04b9738c447fb48a08c0a6c00eca P 404270aa756047a288291fd808377592: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:14:50.792745 18718 tablet_bootstrap.cc:492] T 428f04b9738c447fb48a08c0a6c00eca P f3e90ba48a6a44b5bb4dfbc06888f1c3: No bootstrap required, opened a new log
I20260810 07:14:50.792817 18718 ts_tablet_manager.cc:1397] T 428f04b9738c447fb48a08c0a6c00eca P f3e90ba48a6a44b5bb4dfbc06888f1c3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:14:50.793990 18719 raft_consensus.cc:348] T 428f04b9738c447fb48a08c0a6c00eca P 404270aa756047a288291fd808377592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } }
I20260810 07:14:50.794148 18719 raft_consensus.cc:374] T 428f04b9738c447fb48a08c0a6c00eca P 404270aa756047a288291fd808377592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:14:50.794203 18719 raft_consensus.cc:729] T 428f04b9738c447fb48a08c0a6c00eca P 404270aa756047a288291fd808377592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 404270aa756047a288291fd808377592, State: Initialized, Role: FOLLOWER
I20260810 07:14:50.794315 18719 consensus_queue.cc:260] T 428f04b9738c447fb48a08c0a6c00eca P 404270aa756047a288291fd808377592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } }
I20260810 07:14:50.794471 18717 raft_consensus.cc:348] T 428f04b9738c447fb48a08c0a6c00eca P cd1618f314b441a99f3374fa93174aa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } }
I20260810 07:14:50.794589 18719 ts_tablet_manager.cc:1428] T 428f04b9738c447fb48a08c0a6c00eca P 404270aa756047a288291fd808377592: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 07:14:50.794602 18388 heartbeater.cc:503] Master 127.15.217.126:41785 was elected leader, sending a full tablet report...
I20260810 07:14:50.794629 18717 raft_consensus.cc:374] T 428f04b9738c447fb48a08c0a6c00eca P cd1618f314b441a99f3374fa93174aa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:14:50.794667 18717 raft_consensus.cc:729] T 428f04b9738c447fb48a08c0a6c00eca P cd1618f314b441a99f3374fa93174aa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd1618f314b441a99f3374fa93174aa1, State: Initialized, Role: FOLLOWER
I20260810 07:14:50.794811 18717 consensus_queue.cc:260] T 428f04b9738c447fb48a08c0a6c00eca P cd1618f314b441a99f3374fa93174aa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } }
I20260810 07:14:50.794792 18718 raft_consensus.cc:348] T 428f04b9738c447fb48a08c0a6c00eca P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } }
I20260810 07:14:50.794935 18718 raft_consensus.cc:374] T 428f04b9738c447fb48a08c0a6c00eca P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:14:50.794976 18718 raft_consensus.cc:729] T 428f04b9738c447fb48a08c0a6c00eca P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e90ba48a6a44b5bb4dfbc06888f1c3, State: Initialized, Role: FOLLOWER
I20260810 07:14:50.795091 18717 ts_tablet_manager.cc:1428] T 428f04b9738c447fb48a08c0a6c00eca P cd1618f314b441a99f3374fa93174aa1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:14:50.795122 18718 consensus_queue.cc:260] T 428f04b9738c447fb48a08c0a6c00eca P f3e90ba48a6a44b5bb4dfbc06888f1c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } }
I20260810 07:14:50.795223 18693 heartbeater.cc:503] Master 127.15.217.126:41785 was elected leader, sending a full tablet report...
I20260810 07:14:50.795399 18718 ts_tablet_manager.cc:1428] T 428f04b9738c447fb48a08c0a6c00eca P f3e90ba48a6a44b5bb4dfbc06888f1c3: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 07:14:50.795483 18539 heartbeater.cc:503] Master 127.15.217.126:41785 was elected leader, sending a full tablet report...
W20260810 07:14:50.828949 18540 tablet.cc:2369] T 428f04b9738c447fb48a08c0a6c00eca P f3e90ba48a6a44b5bb4dfbc06888f1c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:14:50.848423 18725 raft_consensus.cc:482] T 428f04b9738c447fb48a08c0a6c00eca P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:14:50.848603 18725 raft_consensus.cc:504] T 428f04b9738c447fb48a08c0a6c00eca P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } }
I20260810 07:14:50.848937 18725 leader_election.cc:288] T 428f04b9738c447fb48a08c0a6c00eca P f3e90ba48a6a44b5bb4dfbc06888f1c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 404270aa756047a288291fd808377592 (127.15.217.65:34161), cd1618f314b441a99f3374fa93174aa1 (127.15.217.67:34305)
I20260810 07:14:50.852558 18714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:50.849336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:59005 (local address 127.15.217.65:34161)
0810 07:14:50.849376 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:50.849378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:50.849387 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:14:50.849419 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:50.849420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:50.849433 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:50.849521 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:14:50.849736 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.850295 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.851776 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.852148 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.852209 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:50.852236 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:50.852406 (+ 170us) server_negotiation.cc:299] Negotiation successful
0810 07:14:50.852474 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260810 07:14:50.852582 18713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:50.849261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:49131 (local address 127.15.217.67:34305)
0810 07:14:50.849321 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:50.849323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:50.849438 (+ 115us) server_negotiation.cc:407] Connection header received
0810 07:14:50.849504 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:50.849505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:50.849519 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:50.849573 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:14:50.849770 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.850355 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.851808 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.852162 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.852252 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:50.852260 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:50.852473 (+ 213us) server_negotiation.cc:299] Negotiation successful
0810 07:14:50.852522 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 07:14:50.852578 18426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:50.849161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:34161 (local address 127.15.217.66:59005)
0810 07:14:50.849231 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:50.849244 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:50.849297 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:50.849569 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:50.849570 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:50.849573 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:14:50.849663 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:50.849666 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.850373 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.850374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:50.851677 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:50.851680 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.852236 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.852237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:50.852325 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:50.852329 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:14:50.852379 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 07:14:50.852446 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":64}
I20260810 07:14:50.852608 18726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:50.849162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:34305 (local address 127.15.217.66:49131)
0810 07:14:50.849362 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:50.849371 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:50.849445 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:50.849590 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:50.849591 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:50.849593 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:14:50.849694 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:50.849696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.850451 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.850452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:50.851677 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:50.851680 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.852275 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.852276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:50.852350 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:50.852354 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:14:50.852438 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 07:14:50.852498 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":2,"thread_start_us":119,"threads_started":1}
I20260810 07:14:50.853121 18340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "428f04b9738c447fb48a08c0a6c00eca" candidate_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "404270aa756047a288291fd808377592" is_pre_election: true
I20260810 07:14:50.853183 18640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "428f04b9738c447fb48a08c0a6c00eca" candidate_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd1618f314b441a99f3374fa93174aa1" is_pre_election: true
I20260810 07:14:50.853318 18340 raft_consensus.cc:2454] T 428f04b9738c447fb48a08c0a6c00eca P 404270aa756047a288291fd808377592 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3e90ba48a6a44b5bb4dfbc06888f1c3 in term 0.
I20260810 07:14:50.853353 18640 raft_consensus.cc:2454] T 428f04b9738c447fb48a08c0a6c00eca P cd1618f314b441a99f3374fa93174aa1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3e90ba48a6a44b5bb4dfbc06888f1c3 in term 0.
I20260810 07:14:50.853574 18417 leader_election.cc:302] T 428f04b9738c447fb48a08c0a6c00eca P f3e90ba48a6a44b5bb4dfbc06888f1c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd1618f314b441a99f3374fa93174aa1, f3e90ba48a6a44b5bb4dfbc06888f1c3; no voters:
I20260810 07:14:50.853713 18725 raft_consensus.cc:2781] T 428f04b9738c447fb48a08c0a6c00eca P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:14:50.853770 18725 raft_consensus.cc:482] T 428f04b9738c447fb48a08c0a6c00eca P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:14:50.853797 18725 raft_consensus.cc:3037] T 428f04b9738c447fb48a08c0a6c00eca P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:14:50.854804 18725 raft_consensus.cc:504] T 428f04b9738c447fb48a08c0a6c00eca P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } }
I20260810 07:14:50.854954 18725 leader_election.cc:288] T 428f04b9738c447fb48a08c0a6c00eca P f3e90ba48a6a44b5bb4dfbc06888f1c3 [CANDIDATE]: Term 1 election: Requested vote from peers 404270aa756047a288291fd808377592 (127.15.217.65:34161), cd1618f314b441a99f3374fa93174aa1 (127.15.217.67:34305)
I20260810 07:14:50.855130 18340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "428f04b9738c447fb48a08c0a6c00eca" candidate_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "404270aa756047a288291fd808377592"
I20260810 07:14:50.855130 18640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "428f04b9738c447fb48a08c0a6c00eca" candidate_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd1618f314b441a99f3374fa93174aa1"
I20260810 07:14:50.855222 18640 raft_consensus.cc:3037] T 428f04b9738c447fb48a08c0a6c00eca P cd1618f314b441a99f3374fa93174aa1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:14:50.855222 18340 raft_consensus.cc:3037] T 428f04b9738c447fb48a08c0a6c00eca P 404270aa756047a288291fd808377592 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:14:50.856192 18640 raft_consensus.cc:2454] T 428f04b9738c447fb48a08c0a6c00eca P cd1618f314b441a99f3374fa93174aa1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3e90ba48a6a44b5bb4dfbc06888f1c3 in term 1.
I20260810 07:14:50.856190 18340 raft_consensus.cc:2454] T 428f04b9738c447fb48a08c0a6c00eca P 404270aa756047a288291fd808377592 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3e90ba48a6a44b5bb4dfbc06888f1c3 in term 1.
I20260810 07:14:50.856408 18417 leader_election.cc:302] T 428f04b9738c447fb48a08c0a6c00eca P f3e90ba48a6a44b5bb4dfbc06888f1c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd1618f314b441a99f3374fa93174aa1, f3e90ba48a6a44b5bb4dfbc06888f1c3; no voters:
I20260810 07:14:50.856530 18725 raft_consensus.cc:2781] T 428f04b9738c447fb48a08c0a6c00eca P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:14:50.856729 18725 raft_consensus.cc:686] T 428f04b9738c447fb48a08c0a6c00eca P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 1 LEADER]: Becoming Leader. State: Replica: f3e90ba48a6a44b5bb4dfbc06888f1c3, State: Running, Role: LEADER
I20260810 07:14:50.856828 18725 consensus_queue.cc:237] T 428f04b9738c447fb48a08c0a6c00eca P f3e90ba48a6a44b5bb4dfbc06888f1c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } }
I20260810 07:14:50.857671 18162 catalog_manager.cc:5495] T 428f04b9738c447fb48a08c0a6c00eca P f3e90ba48a6a44b5bb4dfbc06888f1c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3e90ba48a6a44b5bb4dfbc06888f1c3 (127.15.217.66). New cstate: current_term: 1 leader_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } health_report { overall_health: HEALTHY } } }
Aug 10 07:14:50 dist-test-slave-10pc krb5kdc[18115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346090, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:14:50.896279 18734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:50.889212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41785 (local address 127.0.0.1:58924)
0810 07:14:50.889410 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:50.889422 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:50.889494 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:50.889624 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:50.889625 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:50.889952 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:14:50.890006 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:50.890008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.890653 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.890655 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:50.891238 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:50.891253 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.891708 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.891710 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:50.891776 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:50.892318 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:14:50.892323 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:14:50.893853 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:14:50.895337 (+ 1484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:50.895344 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:50.895347 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:50.895549 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:50.895735 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:50.895736 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:50.895737 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:50.895795 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:50.896011 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:14:50.896018 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:14:50.896059 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 07:14:50.896113 (+ 54us) client_negotiation.cc:240] Negotiation successful
0810 07:14:50.896200 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":110,"threads_started":1}
I20260810 07:14:50.896391 18218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:50.889258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58924 (local address 127.15.217.126:41785)
0810 07:14:50.889334 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:50.889337 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:50.889501 (+ 164us) server_negotiation.cc:407] Connection header received
0810 07:14:50.889549 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:50.889550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:50.889559 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:50.889601 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:50.890076 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.890571 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.891334 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.891612 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.891693 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:50.893942 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:50.893943 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:50.893944 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:50.893951 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:50.895251 (+ 1300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:50.895614 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:50.895615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:50.895615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:50.895674 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:50.895847 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:50.895848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:50.895848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:50.895954 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:50.896013 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:50.896224 (+ 211us) server_negotiation.cc:299] Negotiation successful
0810 07:14:50.896302 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 07:14:50.898602 18162 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:58924:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 07:14:50.898850 18162 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:14:50.900514 18317 tablet_service.cc:1467] Processing CreateTablet for tablet 4b14b405734b4844ba8e1e235d59bf01 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bb98ab4b429b4f4d934dd127ea183495]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 07:14:50.900529 18468 tablet_service.cc:1467] Processing CreateTablet for tablet 4b14b405734b4844ba8e1e235d59bf01 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bb98ab4b429b4f4d934dd127ea183495]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 07:14:50.900554 18619 tablet_service.cc:1467] Processing CreateTablet for tablet 4b14b405734b4844ba8e1e235d59bf01 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bb98ab4b429b4f4d934dd127ea183495]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 07:14:50.900712 18619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b14b405734b4844ba8e1e235d59bf01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:14:50.900717 18468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b14b405734b4844ba8e1e235d59bf01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:14:50.900729 18317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b14b405734b4844ba8e1e235d59bf01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:14:50.901921 18717 tablet_bootstrap.cc:492] T 4b14b405734b4844ba8e1e235d59bf01 P cd1618f314b441a99f3374fa93174aa1: Bootstrap starting.
I20260810 07:14:50.902352 18718 tablet_bootstrap.cc:492] T 4b14b405734b4844ba8e1e235d59bf01 P f3e90ba48a6a44b5bb4dfbc06888f1c3: Bootstrap starting.
I20260810 07:14:50.902544 18717 tablet_bootstrap.cc:654] T 4b14b405734b4844ba8e1e235d59bf01 P cd1618f314b441a99f3374fa93174aa1: Neither blocks nor log segments found. Creating new log.
I20260810 07:14:50.902755 18718 tablet_bootstrap.cc:654] T 4b14b405734b4844ba8e1e235d59bf01 P f3e90ba48a6a44b5bb4dfbc06888f1c3: Neither blocks nor log segments found. Creating new log.
I20260810 07:14:50.903142 18719 tablet_bootstrap.cc:492] T 4b14b405734b4844ba8e1e235d59bf01 P 404270aa756047a288291fd808377592: Bootstrap starting.
I20260810 07:14:50.903278 18717 tablet_bootstrap.cc:492] T 4b14b405734b4844ba8e1e235d59bf01 P cd1618f314b441a99f3374fa93174aa1: No bootstrap required, opened a new log
I20260810 07:14:50.903329 18717 ts_tablet_manager.cc:1397] T 4b14b405734b4844ba8e1e235d59bf01 P cd1618f314b441a99f3374fa93174aa1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:14:50.903467 18717 raft_consensus.cc:348] T 4b14b405734b4844ba8e1e235d59bf01 P cd1618f314b441a99f3374fa93174aa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } }
I20260810 07:14:50.903535 18717 raft_consensus.cc:374] T 4b14b405734b4844ba8e1e235d59bf01 P cd1618f314b441a99f3374fa93174aa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:14:50.903548 18717 raft_consensus.cc:729] T 4b14b405734b4844ba8e1e235d59bf01 P cd1618f314b441a99f3374fa93174aa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd1618f314b441a99f3374fa93174aa1, State: Initialized, Role: FOLLOWER
I20260810 07:14:50.903530 18718 tablet_bootstrap.cc:492] T 4b14b405734b4844ba8e1e235d59bf01 P f3e90ba48a6a44b5bb4dfbc06888f1c3: No bootstrap required, opened a new log
I20260810 07:14:50.903584 18718 ts_tablet_manager.cc:1397] T 4b14b405734b4844ba8e1e235d59bf01 P f3e90ba48a6a44b5bb4dfbc06888f1c3: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 07:14:50.903607 18717 consensus_queue.cc:260] T 4b14b405734b4844ba8e1e235d59bf01 P cd1618f314b441a99f3374fa93174aa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } }
I20260810 07:14:50.903725 18717 ts_tablet_manager.cc:1428] T 4b14b405734b4844ba8e1e235d59bf01 P cd1618f314b441a99f3374fa93174aa1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 07:14:50.903733 18718 raft_consensus.cc:348] T 4b14b405734b4844ba8e1e235d59bf01 P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } }
I20260810 07:14:50.903837 18718 raft_consensus.cc:374] T 4b14b405734b4844ba8e1e235d59bf01 P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:14:50.903864 18718 raft_consensus.cc:729] T 4b14b405734b4844ba8e1e235d59bf01 P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e90ba48a6a44b5bb4dfbc06888f1c3, State: Initialized, Role: FOLLOWER
I20260810 07:14:50.903867 18719 tablet_bootstrap.cc:654] T 4b14b405734b4844ba8e1e235d59bf01 P 404270aa756047a288291fd808377592: Neither blocks nor log segments found. Creating new log.
I20260810 07:14:50.903801 18724 tablet_replica.cc:439] T 4b14b405734b4844ba8e1e235d59bf01 P cd1618f314b441a99f3374fa93174aa1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } } }
I20260810 07:14:50.903915 18724 tablet_replica.cc:442] T 4b14b405734b4844ba8e1e235d59bf01 P cd1618f314b441a99f3374fa93174aa1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:14:50.903916 18718 consensus_queue.cc:260] T 4b14b405734b4844ba8e1e235d59bf01 P f3e90ba48a6a44b5bb4dfbc06888f1c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } }
I20260810 07:14:50.904021 18718 ts_tablet_manager.cc:1428] T 4b14b405734b4844ba8e1e235d59bf01 P f3e90ba48a6a44b5bb4dfbc06888f1c3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:50.904268 18725 tablet_replica.cc:439] T 4b14b405734b4844ba8e1e235d59bf01 P f3e90ba48a6a44b5bb4dfbc06888f1c3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } } }
I20260810 07:14:50.904364 18725 tablet_replica.cc:442] T 4b14b405734b4844ba8e1e235d59bf01 P f3e90ba48a6a44b5bb4dfbc06888f1c3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:14:50.904614 18719 tablet_bootstrap.cc:492] T 4b14b405734b4844ba8e1e235d59bf01 P 404270aa756047a288291fd808377592: No bootstrap required, opened a new log
I20260810 07:14:50.904663 18719 ts_tablet_manager.cc:1397] T 4b14b405734b4844ba8e1e235d59bf01 P 404270aa756047a288291fd808377592: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 07:14:50.904817 18719 raft_consensus.cc:348] T 4b14b405734b4844ba8e1e235d59bf01 P 404270aa756047a288291fd808377592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } }
I20260810 07:14:50.904884 18719 raft_consensus.cc:374] T 4b14b405734b4844ba8e1e235d59bf01 P 404270aa756047a288291fd808377592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:14:50.904901 18719 raft_consensus.cc:729] T 4b14b405734b4844ba8e1e235d59bf01 P 404270aa756047a288291fd808377592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 404270aa756047a288291fd808377592, State: Initialized, Role: FOLLOWER
I20260810 07:14:50.904963 18719 consensus_queue.cc:260] T 4b14b405734b4844ba8e1e235d59bf01 P 404270aa756047a288291fd808377592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } }
I20260810 07:14:50.905046 18719 ts_tablet_manager.cc:1428] T 4b14b405734b4844ba8e1e235d59bf01 P 404270aa756047a288291fd808377592: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 07:14:50.905176 18723 tablet_replica.cc:439] T 4b14b405734b4844ba8e1e235d59bf01 P 404270aa756047a288291fd808377592: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } } }
I20260810 07:14:50.905251 18723 tablet_replica.cc:442] T 4b14b405734b4844ba8e1e235d59bf01 P 404270aa756047a288291fd808377592: This TxnStatusTablet replica's current role is: FOLLOWER
W20260810 07:14:50.905822 18389 tablet.cc:2369] T 428f04b9738c447fb48a08c0a6c00eca P 404270aa756047a288291fd808377592: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:14:50.950265 18725 raft_consensus.cc:482] T 4b14b405734b4844ba8e1e235d59bf01 P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:14:50.950428 18725 raft_consensus.cc:504] T 4b14b405734b4844ba8e1e235d59bf01 P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } }
I20260810 07:14:50.950635 18725 leader_election.cc:288] T 4b14b405734b4844ba8e1e235d59bf01 P f3e90ba48a6a44b5bb4dfbc06888f1c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 404270aa756047a288291fd808377592 (127.15.217.65:34161), cd1618f314b441a99f3374fa93174aa1 (127.15.217.67:34305)
I20260810 07:14:50.950825 18340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b14b405734b4844ba8e1e235d59bf01" candidate_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "404270aa756047a288291fd808377592" is_pre_election: true
I20260810 07:14:50.950891 18640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b14b405734b4844ba8e1e235d59bf01" candidate_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd1618f314b441a99f3374fa93174aa1" is_pre_election: true
I20260810 07:14:50.950969 18340 raft_consensus.cc:2454] T 4b14b405734b4844ba8e1e235d59bf01 P 404270aa756047a288291fd808377592 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3e90ba48a6a44b5bb4dfbc06888f1c3 in term 0.
I20260810 07:14:50.950977 18640 raft_consensus.cc:2454] T 4b14b405734b4844ba8e1e235d59bf01 P cd1618f314b441a99f3374fa93174aa1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3e90ba48a6a44b5bb4dfbc06888f1c3 in term 0.
I20260810 07:14:50.951220 18419 leader_election.cc:302] T 4b14b405734b4844ba8e1e235d59bf01 P f3e90ba48a6a44b5bb4dfbc06888f1c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 404270aa756047a288291fd808377592, f3e90ba48a6a44b5bb4dfbc06888f1c3; no voters:
I20260810 07:14:50.951390 18725 raft_consensus.cc:2781] T 4b14b405734b4844ba8e1e235d59bf01 P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:14:50.951434 18725 raft_consensus.cc:482] T 4b14b405734b4844ba8e1e235d59bf01 P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:14:50.951453 18725 raft_consensus.cc:3037] T 4b14b405734b4844ba8e1e235d59bf01 P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:14:50.952143 18725 raft_consensus.cc:504] T 4b14b405734b4844ba8e1e235d59bf01 P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } }
I20260810 07:14:50.952374 18725 leader_election.cc:288] T 4b14b405734b4844ba8e1e235d59bf01 P f3e90ba48a6a44b5bb4dfbc06888f1c3 [CANDIDATE]: Term 1 election: Requested vote from peers 404270aa756047a288291fd808377592 (127.15.217.65:34161), cd1618f314b441a99f3374fa93174aa1 (127.15.217.67:34305)
I20260810 07:14:50.952610 18640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b14b405734b4844ba8e1e235d59bf01" candidate_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd1618f314b441a99f3374fa93174aa1"
I20260810 07:14:50.952692 18640 raft_consensus.cc:3037] T 4b14b405734b4844ba8e1e235d59bf01 P cd1618f314b441a99f3374fa93174aa1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:14:50.953392 18640 raft_consensus.cc:2454] T 4b14b405734b4844ba8e1e235d59bf01 P cd1618f314b441a99f3374fa93174aa1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3e90ba48a6a44b5bb4dfbc06888f1c3 in term 1.
I20260810 07:14:50.953558 18340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b14b405734b4844ba8e1e235d59bf01" candidate_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "404270aa756047a288291fd808377592"
I20260810 07:14:50.953584 18417 leader_election.cc:302] T 4b14b405734b4844ba8e1e235d59bf01 P f3e90ba48a6a44b5bb4dfbc06888f1c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd1618f314b441a99f3374fa93174aa1, f3e90ba48a6a44b5bb4dfbc06888f1c3; no voters:
I20260810 07:14:50.953665 18340 raft_consensus.cc:3037] T 4b14b405734b4844ba8e1e235d59bf01 P 404270aa756047a288291fd808377592 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:14:50.953761 18725 raft_consensus.cc:2781] T 4b14b405734b4844ba8e1e235d59bf01 P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:14:50.953832 18725 raft_consensus.cc:686] T 4b14b405734b4844ba8e1e235d59bf01 P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 1 LEADER]: Becoming Leader. State: Replica: f3e90ba48a6a44b5bb4dfbc06888f1c3, State: Running, Role: LEADER
I20260810 07:14:50.953887 18725 consensus_queue.cc:237] T 4b14b405734b4844ba8e1e235d59bf01 P f3e90ba48a6a44b5bb4dfbc06888f1c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } }
I20260810 07:14:50.954425 18340 raft_consensus.cc:2454] T 4b14b405734b4844ba8e1e235d59bf01 P 404270aa756047a288291fd808377592 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3e90ba48a6a44b5bb4dfbc06888f1c3 in term 1.
I20260810 07:14:50.954434 18727 tablet_replica.cc:439] T 4b14b405734b4844ba8e1e235d59bf01 P f3e90ba48a6a44b5bb4dfbc06888f1c3: TxnStatusTablet state changed. Reason: New leader f3e90ba48a6a44b5bb4dfbc06888f1c3. Latest consensus state: current_term: 1 leader_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } } }
I20260810 07:14:50.954527 18727 tablet_replica.cc:442] T 4b14b405734b4844ba8e1e235d59bf01 P f3e90ba48a6a44b5bb4dfbc06888f1c3: This TxnStatusTablet replica's current role is: LEADER
I20260810 07:14:50.954869 18739 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 07:14:50.955071 18739 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 07:14:50.955120 18163 catalog_manager.cc:5495] T 4b14b405734b4844ba8e1e235d59bf01 P f3e90ba48a6a44b5bb4dfbc06888f1c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3e90ba48a6a44b5bb4dfbc06888f1c3 (127.15.217.66). New cstate: current_term: 1 leader_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } health_report { overall_health: UNKNOWN } } }
I20260810 07:14:50.964064 18716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:50.961530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58220 (local address 127.15.217.66:33449)
0810 07:14:50.961594 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:50.961597 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:50.961606 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:14:50.961683 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:50.961684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:50.961704 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:50.961764 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:14:50.961901 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.962395 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.963161 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.963378 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.963453 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:50.963720 (+ 267us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:14:50.963828 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:14:50.963922 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:50.963967 (+ 45us) server_negotiation.cc:299] Negotiation successful
0810 07:14:50.963991 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 07:14:50.964095 18734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:50.961440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:33449 (local address 127.0.0.1:58220)
0810 07:14:50.961513 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:50.961529 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:50.961593 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:50.961773 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:50.961774 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:50.961775 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:14:50.961834 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:50.961837 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.962480 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.962482 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:50.963071 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:50.963073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.963454 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.963455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:50.963531 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:50.963548 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:14:50.963898 (+ 350us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:14:50.963910 (+ 12us) client_negotiation.cc:769] Sending connection context
0810 07:14:50.963959 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 07:14:50.964019 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260810 07:14:50.966087 18340 raft_consensus.cc:1261] T 4b14b405734b4844ba8e1e235d59bf01 P 404270aa756047a288291fd808377592 [term 1 FOLLOWER]: Refusing update from remote peer f3e90ba48a6a44b5bb4dfbc06888f1c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:14:50.966123 18640 raft_consensus.cc:1261] T 4b14b405734b4844ba8e1e235d59bf01 P cd1618f314b441a99f3374fa93174aa1 [term 1 FOLLOWER]: Refusing update from remote peer f3e90ba48a6a44b5bb4dfbc06888f1c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:14:50.966449 18727 consensus_queue.cc:1035] T 4b14b405734b4844ba8e1e235d59bf01 P f3e90ba48a6a44b5bb4dfbc06888f1c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:14:50.966588 18725 consensus_queue.cc:1035] T 4b14b405734b4844ba8e1e235d59bf01 P f3e90ba48a6a44b5bb4dfbc06888f1c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:14:50.967866 18723 tablet_replica.cc:439] T 4b14b405734b4844ba8e1e235d59bf01 P 404270aa756047a288291fd808377592: TxnStatusTablet state changed. Reason: New leader f3e90ba48a6a44b5bb4dfbc06888f1c3. Latest consensus state: current_term: 1 leader_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } } }
I20260810 07:14:50.967963 18723 tablet_replica.cc:442] T 4b14b405734b4844ba8e1e235d59bf01 P 404270aa756047a288291fd808377592: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:14:50.967890 18724 tablet_replica.cc:439] T 4b14b405734b4844ba8e1e235d59bf01 P cd1618f314b441a99f3374fa93174aa1: TxnStatusTablet state changed. Reason: New leader f3e90ba48a6a44b5bb4dfbc06888f1c3. Latest consensus state: current_term: 1 leader_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } } }
I20260810 07:14:50.968009 18724 tablet_replica.cc:442] T 4b14b405734b4844ba8e1e235d59bf01 P cd1618f314b441a99f3374fa93174aa1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:14:50.969182 18727 tablet_replica.cc:439] T 4b14b405734b4844ba8e1e235d59bf01 P f3e90ba48a6a44b5bb4dfbc06888f1c3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } } }
I20260810 07:14:50.969280 18727 tablet_replica.cc:442] T 4b14b405734b4844ba8e1e235d59bf01 P f3e90ba48a6a44b5bb4dfbc06888f1c3: This TxnStatusTablet replica's current role is: LEADER
I20260810 07:14:50.969336 18727 tablet_replica.cc:439] T 4b14b405734b4844ba8e1e235d59bf01 P f3e90ba48a6a44b5bb4dfbc06888f1c3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } } }
I20260810 07:14:50.969383 18727 tablet_replica.cc:442] T 4b14b405734b4844ba8e1e235d59bf01 P f3e90ba48a6a44b5bb4dfbc06888f1c3: This TxnStatusTablet replica's current role is: LEADER
I20260810 07:14:50.969480 18723 tablet_replica.cc:439] T 4b14b405734b4844ba8e1e235d59bf01 P 404270aa756047a288291fd808377592: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } } }
I20260810 07:14:50.969561 18723 tablet_replica.cc:442] T 4b14b405734b4844ba8e1e235d59bf01 P 404270aa756047a288291fd808377592: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:14:50.969949 18742 mvcc.cc:205] Tried to move back new op lower bound from 7316873588594556928 to 7316873588548096000. Current Snapshot: MvccSnapshot[applied={T|T < 7316873588594556928}]
I20260810 07:14:50.970074 18743 mvcc.cc:205] Tried to move back new op lower bound from 7316873588594556928 to 7316873588548096000. Current Snapshot: MvccSnapshot[applied={T|T < 7316873588594556928 or (T in {7316873588594556928})}]
I20260810 07:14:50.970198 18724 tablet_replica.cc:439] T 4b14b405734b4844ba8e1e235d59bf01 P cd1618f314b441a99f3374fa93174aa1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } } }
I20260810 07:14:50.970289 18724 tablet_replica.cc:442] T 4b14b405734b4844ba8e1e235d59bf01 P cd1618f314b441a99f3374fa93174aa1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:14:50.970468 18744 mvcc.cc:205] Tried to move back new op lower bound from 7316873588594556928 to 7316873588548096000. Current Snapshot: MvccSnapshot[applied={T|T < 7316873588594556928}]
I20260810 07:14:50.974749 18716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:50.972327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58232 (local address 127.15.217.66:33449)
0810 07:14:50.972374 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:50.972376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:50.972386 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:14:50.972481 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:50.972482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:50.972493 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:50.972560 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:14:50.972707 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.973179 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.974026 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.974222 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.974304 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:50.974472 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:14:50.974516 (+ 44us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:14:50.974581 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:50.974638 (+ 57us) server_negotiation.cc:299] Negotiation successful
0810 07:14:50.974664 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 07:14:50.979674 18716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:50.977492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:38747 (local address 127.15.217.66:33449)
0810 07:14:50.977538 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:50.977541 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:50.977549 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:14:50.977607 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:50.977608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:50.977617 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:50.977668 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:14:50.977810 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.978246 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.978993 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.979188 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.979258 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:50.979441 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:14:50.979472 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:14:50.979514 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:50.979569 (+ 55us) server_negotiation.cc:299] Negotiation successful
0810 07:14:50.979587 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":31}
I20260810 07:14:50.979727 18548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:50.977417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:33449 (local address 127.15.217.66:38747)
0810 07:14:50.977491 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:50.977502 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:50.977548 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:50.977678 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:50.977679 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:50.977680 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:14:50.977734 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:50.977735 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.978327 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.978328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:50.978900 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:50.978901 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:50.979256 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:50.979257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:50.979320 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:50.979339 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:14:50.979525 (+ 186us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:14:50.979526 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 07:14:50.979568 (+ 42us) client_negotiation.cc:240] Negotiation successful
0810 07:14:50.979610 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 07:14:50.982360 18640 raft_consensus.cc:1261] T 428f04b9738c447fb48a08c0a6c00eca P cd1618f314b441a99f3374fa93174aa1 [term 1 FOLLOWER]: Refusing update from remote peer f3e90ba48a6a44b5bb4dfbc06888f1c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:14:50.982363 18340 raft_consensus.cc:1261] T 428f04b9738c447fb48a08c0a6c00eca P 404270aa756047a288291fd808377592 [term 1 FOLLOWER]: Refusing update from remote peer f3e90ba48a6a44b5bb4dfbc06888f1c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:14:50.982586 18727 consensus_queue.cc:1035] T 428f04b9738c447fb48a08c0a6c00eca P f3e90ba48a6a44b5bb4dfbc06888f1c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:14:50.982764 18725 consensus_queue.cc:1035] T 428f04b9738c447fb48a08c0a6c00eca P f3e90ba48a6a44b5bb4dfbc06888f1c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:14:50.998309 18163 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:14:50.998454 18163 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:14:50.999300 18163 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1b66515d7fd64e6dbd7495ec1558830f: Sending DeleteTablet for 3 replicas of tablet 428f04b9738c447fb48a08c0a6c00eca
I20260810 07:14:50.999651 18317 tablet_service.cc:1514] Processing DeleteTablet for tablet 428f04b9738c447fb48a08c0a6c00eca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:14:50 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:34406
I20260810 07:14:50.999701 18619 tablet_service.cc:1514] Processing DeleteTablet for tablet 428f04b9738c447fb48a08c0a6c00eca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:14:50 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:37220
I20260810 07:14:51.000080 18758 tablet_replica.cc:330] T 428f04b9738c447fb48a08c0a6c00eca P 404270aa756047a288291fd808377592: stopping tablet replica
I20260810 07:14:51.000062 18467 tablet_service.cc:1514] Processing DeleteTablet for tablet 428f04b9738c447fb48a08c0a6c00eca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:14:50 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:58216
I20260810 07:14:51.000052 18163 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58920:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:14:51.000211 18758 raft_consensus.cc:2229] T 428f04b9738c447fb48a08c0a6c00eca P 404270aa756047a288291fd808377592 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260810 07:14:51.000226 18163 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:14:51.000270 18758 raft_consensus.cc:2258] T 428f04b9738c447fb48a08c0a6c00eca P 404270aa756047a288291fd808377592 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:14:51.000633 18760 tablet_replica.cc:330] T 428f04b9738c447fb48a08c0a6c00eca P f3e90ba48a6a44b5bb4dfbc06888f1c3: stopping tablet replica
I20260810 07:14:51.000818 18760 raft_consensus.cc:2229] T 428f04b9738c447fb48a08c0a6c00eca P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:14:51.000892 18758 ts_tablet_manager.cc:1905] T 428f04b9738c447fb48a08c0a6c00eca P 404270aa756047a288291fd808377592: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:14:51.001453 18760 raft_consensus.cc:2258] T 428f04b9738c447fb48a08c0a6c00eca P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:14:51.001734 18467 tablet_service.cc:1467] Processing CreateTablet for tablet 94e30358b27845c2b4d22a41947e028b (DEFAULT_TABLE table=test-table [id=abe97e92904b49f5ba7a1a873c9546dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:14:51.001884 18467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94e30358b27845c2b4d22a41947e028b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:14:51.002965 18758 ts_tablet_manager.cc:1918] T 428f04b9738c447fb48a08c0a6c00eca P 404270aa756047a288291fd808377592: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:14:51.003057 18758 log.cc:1196] T 428f04b9738c447fb48a08c0a6c00eca P 404270aa756047a288291fd808377592: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-0/wal/wals/428f04b9738c447fb48a08c0a6c00eca
I20260810 07:14:51.003329 18758 ts_tablet_manager.cc:1939] T 428f04b9738c447fb48a08c0a6c00eca P 404270aa756047a288291fd808377592: Deleting consensus metadata
I20260810 07:14:51.003419 18718 tablet_bootstrap.cc:492] T 94e30358b27845c2b4d22a41947e028b P f3e90ba48a6a44b5bb4dfbc06888f1c3: Bootstrap starting.
I20260810 07:14:51.003453 18619 tablet_service.cc:1467] Processing CreateTablet for tablet 94e30358b27845c2b4d22a41947e028b (DEFAULT_TABLE table=test-table [id=abe97e92904b49f5ba7a1a873c9546dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:14:51.003582 18619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94e30358b27845c2b4d22a41947e028b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:14:51.003839 18718 tablet_bootstrap.cc:654] T 94e30358b27845c2b4d22a41947e028b P f3e90ba48a6a44b5bb4dfbc06888f1c3: Neither blocks nor log segments found. Creating new log.
I20260810 07:14:51.004403 18718 tablet_bootstrap.cc:492] T 94e30358b27845c2b4d22a41947e028b P f3e90ba48a6a44b5bb4dfbc06888f1c3: No bootstrap required, opened a new log
I20260810 07:14:51.004490 18718 ts_tablet_manager.cc:1397] T 94e30358b27845c2b4d22a41947e028b P f3e90ba48a6a44b5bb4dfbc06888f1c3: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 07:14:51.004668 18718 raft_consensus.cc:348] T 94e30358b27845c2b4d22a41947e028b P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } }
I20260810 07:14:51.004784 18718 raft_consensus.cc:374] T 94e30358b27845c2b4d22a41947e028b P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:14:51.004813 18718 raft_consensus.cc:729] T 94e30358b27845c2b4d22a41947e028b P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e90ba48a6a44b5bb4dfbc06888f1c3, State: Initialized, Role: FOLLOWER
I20260810 07:14:51.004863 18718 consensus_queue.cc:260] T 94e30358b27845c2b4d22a41947e028b P f3e90ba48a6a44b5bb4dfbc06888f1c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } }
I20260810 07:14:51.004972 18718 ts_tablet_manager.cc:1428] T 94e30358b27845c2b4d22a41947e028b P f3e90ba48a6a44b5bb4dfbc06888f1c3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:51.005122 18717 tablet_bootstrap.cc:492] T 94e30358b27845c2b4d22a41947e028b P cd1618f314b441a99f3374fa93174aa1: Bootstrap starting.
I20260810 07:14:51.005373 18760 ts_tablet_manager.cc:1905] T 428f04b9738c447fb48a08c0a6c00eca P f3e90ba48a6a44b5bb4dfbc06888f1c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:14:51.005522 18717 tablet_bootstrap.cc:654] T 94e30358b27845c2b4d22a41947e028b P cd1618f314b441a99f3374fa93174aa1: Neither blocks nor log segments found. Creating new log.
I20260810 07:14:51.006098 18317 tablet_service.cc:1467] Processing CreateTablet for tablet 94e30358b27845c2b4d22a41947e028b (DEFAULT_TABLE table=test-table [id=abe97e92904b49f5ba7a1a873c9546dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:14:51.006220 18317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94e30358b27845c2b4d22a41947e028b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:14:51.007225 18760 ts_tablet_manager.cc:1918] T 428f04b9738c447fb48a08c0a6c00eca P f3e90ba48a6a44b5bb4dfbc06888f1c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:14:51.007313 18760 log.cc:1196] T 428f04b9738c447fb48a08c0a6c00eca P f3e90ba48a6a44b5bb4dfbc06888f1c3: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-1/wal/wals/428f04b9738c447fb48a08c0a6c00eca
I20260810 07:14:51.007475 18719 tablet_bootstrap.cc:492] T 94e30358b27845c2b4d22a41947e028b P 404270aa756047a288291fd808377592: Bootstrap starting.
I20260810 07:14:51.007619 18760 ts_tablet_manager.cc:1939] T 428f04b9738c447fb48a08c0a6c00eca P f3e90ba48a6a44b5bb4dfbc06888f1c3: Deleting consensus metadata
I20260810 07:14:51.007691 18148 catalog_manager.cc:4841] TS 404270aa756047a288291fd808377592 (127.15.217.65:34161): tablet 428f04b9738c447fb48a08c0a6c00eca (table test-table [id=df6803d3aa9b44f69b769d3bfa31f4af]) successfully deleted
I20260810 07:14:51.007912 18719 tablet_bootstrap.cc:654] T 94e30358b27845c2b4d22a41947e028b P 404270aa756047a288291fd808377592: Neither blocks nor log segments found. Creating new log.
I20260810 07:14:51.008076 18148 catalog_manager.cc:4841] TS f3e90ba48a6a44b5bb4dfbc06888f1c3 (127.15.217.66:33449): tablet 428f04b9738c447fb48a08c0a6c00eca (table test-table [id=df6803d3aa9b44f69b769d3bfa31f4af]) successfully deleted
I20260810 07:14:51.008531 18717 tablet_bootstrap.cc:492] T 94e30358b27845c2b4d22a41947e028b P cd1618f314b441a99f3374fa93174aa1: No bootstrap required, opened a new log
I20260810 07:14:51.008589 18717 ts_tablet_manager.cc:1397] T 94e30358b27845c2b4d22a41947e028b P cd1618f314b441a99f3374fa93174aa1: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260810 07:14:51.008764 18717 raft_consensus.cc:348] T 94e30358b27845c2b4d22a41947e028b P cd1618f314b441a99f3374fa93174aa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } }
I20260810 07:14:51.008844 18717 raft_consensus.cc:374] T 94e30358b27845c2b4d22a41947e028b P cd1618f314b441a99f3374fa93174aa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:14:51.008867 18717 raft_consensus.cc:729] T 94e30358b27845c2b4d22a41947e028b P cd1618f314b441a99f3374fa93174aa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd1618f314b441a99f3374fa93174aa1, State: Initialized, Role: FOLLOWER
I20260810 07:14:51.008898 18719 tablet_bootstrap.cc:492] T 94e30358b27845c2b4d22a41947e028b P 404270aa756047a288291fd808377592: No bootstrap required, opened a new log
I20260810 07:14:51.008944 18719 ts_tablet_manager.cc:1397] T 94e30358b27845c2b4d22a41947e028b P 404270aa756047a288291fd808377592: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:14:51.008935 18759 tablet_replica.cc:330] T 428f04b9738c447fb48a08c0a6c00eca P cd1618f314b441a99f3374fa93174aa1: stopping tablet replica
I20260810 07:14:51.008935 18717 consensus_queue.cc:260] T 94e30358b27845c2b4d22a41947e028b P cd1618f314b441a99f3374fa93174aa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } }
I20260810 07:14:51.009073 18759 raft_consensus.cc:2229] T 428f04b9738c447fb48a08c0a6c00eca P cd1618f314b441a99f3374fa93174aa1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:14:51.009091 18719 raft_consensus.cc:348] T 94e30358b27845c2b4d22a41947e028b P 404270aa756047a288291fd808377592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } }
I20260810 07:14:51.009133 18717 ts_tablet_manager.cc:1428] T 94e30358b27845c2b4d22a41947e028b P cd1618f314b441a99f3374fa93174aa1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 07:14:51.009176 18719 raft_consensus.cc:374] T 94e30358b27845c2b4d22a41947e028b P 404270aa756047a288291fd808377592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:14:51.009198 18719 raft_consensus.cc:729] T 94e30358b27845c2b4d22a41947e028b P 404270aa756047a288291fd808377592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 404270aa756047a288291fd808377592, State: Initialized, Role: FOLLOWER
I20260810 07:14:51.009239 18719 consensus_queue.cc:260] T 94e30358b27845c2b4d22a41947e028b P 404270aa756047a288291fd808377592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } }
I20260810 07:14:51.009331 18719 ts_tablet_manager.cc:1428] T 94e30358b27845c2b4d22a41947e028b P 404270aa756047a288291fd808377592: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:51.009141 18759 raft_consensus.cc:2258] T 428f04b9738c447fb48a08c0a6c00eca P cd1618f314b441a99f3374fa93174aa1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:14:51.013057 18759 ts_tablet_manager.cc:1905] T 428f04b9738c447fb48a08c0a6c00eca P cd1618f314b441a99f3374fa93174aa1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:14:51.014767 18759 ts_tablet_manager.cc:1918] T 428f04b9738c447fb48a08c0a6c00eca P cd1618f314b441a99f3374fa93174aa1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:14:51.014848 18759 log.cc:1196] T 428f04b9738c447fb48a08c0a6c00eca P cd1618f314b441a99f3374fa93174aa1: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-2/wal/wals/428f04b9738c447fb48a08c0a6c00eca
I20260810 07:14:51.015136 18759 ts_tablet_manager.cc:1939] T 428f04b9738c447fb48a08c0a6c00eca P cd1618f314b441a99f3374fa93174aa1: Deleting consensus metadata
I20260810 07:14:51.015566 18147 catalog_manager.cc:4841] TS cd1618f314b441a99f3374fa93174aa1 (127.15.217.67:34305): tablet 428f04b9738c447fb48a08c0a6c00eca (table test-table [id=df6803d3aa9b44f69b769d3bfa31f4af]) successfully deleted
I20260810 07:14:51.041421 18724 raft_consensus.cc:482] T 94e30358b27845c2b4d22a41947e028b P cd1618f314b441a99f3374fa93174aa1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:14:51.041561 18724 raft_consensus.cc:504] T 94e30358b27845c2b4d22a41947e028b P cd1618f314b441a99f3374fa93174aa1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } }
I20260810 07:14:51.041946 18724 leader_election.cc:288] T 94e30358b27845c2b4d22a41947e028b P cd1618f314b441a99f3374fa93174aa1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 404270aa756047a288291fd808377592 (127.15.217.65:34161), f3e90ba48a6a44b5bb4dfbc06888f1c3 (127.15.217.66:33449)
I20260810 07:14:51.045897 18716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:51.042437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:43505 (local address 127.15.217.66:33449)
0810 07:14:51.042485 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:51.042487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:51.042496 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:14:51.042571 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:51.042572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:51.042582 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:51.042639 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:14:51.042893 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:51.043411 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:51.045084 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:51.045455 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:51.045540 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:51.045554 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:51.045780 (+ 226us) server_negotiation.cc:299] Negotiation successful
0810 07:14:51.045827 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 07:14:51.045893 18765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:51.042258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:33449 (local address 127.15.217.67:43505)
0810 07:14:51.042438 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:51.042448 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:51.042491 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:51.042782 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:51.042783 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:51.042784 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:14:51.042827 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:51.042828 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:51.043491 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:51.043492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:51.045013 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:51.045016 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:51.045553 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:51.045554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:51.045649 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:51.045668 (+ 19us) client_negotiation.cc:769] Sending connection context
0810 07:14:51.045756 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 07:14:51.045802 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":71,"threads_started":1}
I20260810 07:14:51.046072 18714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:51.042219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:41351 (local address 127.15.217.65:34161)
0810 07:14:51.042277 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:51.042279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:51.042293 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:14:51.042517 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:51.042519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:51.042533 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:51.042620 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:14:51.042797 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:51.043540 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:51.045045 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:51.045605 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:51.045694 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:51.045704 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:51.045935 (+ 231us) server_negotiation.cc:299] Negotiation successful
0810 07:14:51.045989 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 07:14:51.046198 18490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94e30358b27845c2b4d22a41947e028b" candidate_uuid: "cd1618f314b441a99f3374fa93174aa1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" is_pre_election: true
I20260810 07:14:51.046357 18490 raft_consensus.cc:2454] T 94e30358b27845c2b4d22a41947e028b P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd1618f314b441a99f3374fa93174aa1 in term 0.
I20260810 07:14:51.046592 18571 leader_election.cc:302] T 94e30358b27845c2b4d22a41947e028b P cd1618f314b441a99f3374fa93174aa1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd1618f314b441a99f3374fa93174aa1, f3e90ba48a6a44b5bb4dfbc06888f1c3; no voters:
I20260810 07:14:51.046702 18578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:51.042134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:34161 (local address 127.15.217.67:41351)
0810 07:14:51.042207 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:51.042220 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:51.042270 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:51.042628 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:51.042629 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:51.042631 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:14:51.042714 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:51.042722 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:51.043634 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:51.043634 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:51.044905 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:51.044908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:51.045700 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:51.045701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:51.045786 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:51.045789 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:14:51.045836 (+ 47us) client_negotiation.cc:240] Negotiation successful
0810 07:14:51.045867 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 07:14:51.046757 18724 raft_consensus.cc:2781] T 94e30358b27845c2b4d22a41947e028b P cd1618f314b441a99f3374fa93174aa1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:14:51.046811 18724 raft_consensus.cc:482] T 94e30358b27845c2b4d22a41947e028b P cd1618f314b441a99f3374fa93174aa1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:14:51.046841 18724 raft_consensus.cc:3037] T 94e30358b27845c2b4d22a41947e028b P cd1618f314b441a99f3374fa93174aa1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:14:51.047785 18724 raft_consensus.cc:504] T 94e30358b27845c2b4d22a41947e028b P cd1618f314b441a99f3374fa93174aa1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } }
I20260810 07:14:51.047936 18724 leader_election.cc:288] T 94e30358b27845c2b4d22a41947e028b P cd1618f314b441a99f3374fa93174aa1 [CANDIDATE]: Term 1 election: Requested vote from peers 404270aa756047a288291fd808377592 (127.15.217.65:34161), f3e90ba48a6a44b5bb4dfbc06888f1c3 (127.15.217.66:33449)
I20260810 07:14:51.048064 18339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94e30358b27845c2b4d22a41947e028b" candidate_uuid: "cd1618f314b441a99f3374fa93174aa1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "404270aa756047a288291fd808377592" is_pre_election: true
I20260810 07:14:51.048163 18339 raft_consensus.cc:2454] T 94e30358b27845c2b4d22a41947e028b P 404270aa756047a288291fd808377592 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd1618f314b441a99f3374fa93174aa1 in term 0.
I20260810 07:14:51.048131 18340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94e30358b27845c2b4d22a41947e028b" candidate_uuid: "cd1618f314b441a99f3374fa93174aa1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "404270aa756047a288291fd808377592"
I20260810 07:14:51.048130 18490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94e30358b27845c2b4d22a41947e028b" candidate_uuid: "cd1618f314b441a99f3374fa93174aa1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3"
I20260810 07:14:51.048209 18340 raft_consensus.cc:3037] T 94e30358b27845c2b4d22a41947e028b P 404270aa756047a288291fd808377592 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:14:51.048229 18490 raft_consensus.cc:3037] T 94e30358b27845c2b4d22a41947e028b P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:14:51.049078 18340 raft_consensus.cc:2454] T 94e30358b27845c2b4d22a41947e028b P 404270aa756047a288291fd808377592 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd1618f314b441a99f3374fa93174aa1 in term 1.
I20260810 07:14:51.049279 18571 leader_election.cc:302] T 94e30358b27845c2b4d22a41947e028b P cd1618f314b441a99f3374fa93174aa1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 404270aa756047a288291fd808377592, cd1618f314b441a99f3374fa93174aa1; no voters:
I20260810 07:14:51.049289 18490 raft_consensus.cc:2454] T 94e30358b27845c2b4d22a41947e028b P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd1618f314b441a99f3374fa93174aa1 in term 1.
I20260810 07:14:51.049367 18724 raft_consensus.cc:2781] T 94e30358b27845c2b4d22a41947e028b P cd1618f314b441a99f3374fa93174aa1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:14:51.049468 18724 raft_consensus.cc:686] T 94e30358b27845c2b4d22a41947e028b P cd1618f314b441a99f3374fa93174aa1 [term 1 LEADER]: Becoming Leader. State: Replica: cd1618f314b441a99f3374fa93174aa1, State: Running, Role: LEADER
I20260810 07:14:51.049546 18724 consensus_queue.cc:237] T 94e30358b27845c2b4d22a41947e028b P cd1618f314b441a99f3374fa93174aa1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } }
I20260810 07:14:51.050400 18163 catalog_manager.cc:5495] T 94e30358b27845c2b4d22a41947e028b P cd1618f314b441a99f3374fa93174aa1 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd1618f314b441a99f3374fa93174aa1 (127.15.217.67). New cstate: current_term: 1 leader_uuid: "cd1618f314b441a99f3374fa93174aa1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } health_report { overall_health: HEALTHY } } }
I20260810 07:14:51.068069 18713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:51.065405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37236 (local address 127.15.217.67:34305)
0810 07:14:51.065448 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:51.065451 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:51.065460 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:14:51.065530 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:51.065531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:51.065554 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:51.065644 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:14:51.065754 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:51.066499 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:51.067254 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:51.067532 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:51.067607 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:51.067726 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:14:51.067858 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:14:51.067933 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:51.067956 (+ 23us) server_negotiation.cc:299] Negotiation successful
0810 07:14:51.067989 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 07:14:51.073711 18716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:51.071053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:55937 (local address 127.15.217.66:33449)
0810 07:14:51.071113 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:51.071115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:51.071123 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:14:51.071146 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:51.071147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:51.071156 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:51.071208 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:14:51.071382 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:51.071833 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:51.072930 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:51.073176 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:51.073234 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:51.073428 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:14:51.073504 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:14:51.073544 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:51.073626 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 07:14:51.073646 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 07:14:51.073765 18700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:51.070761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:33449 (local address 127.15.217.67:55937)
0810 07:14:51.070877 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:51.070892 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:51.070943 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:51.071235 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:51.071236 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:51.071237 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:14:51.071310 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:51.071312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:51.071908 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:51.071909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:51.072832 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:51.072836 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:51.073252 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:51.073253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:51.073328 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:51.073348 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:14:51.073563 (+ 215us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:14:51.073566 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:14:51.073617 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 07:14:51.073668 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 07:14:51.076390 18490 raft_consensus.cc:1261] T 94e30358b27845c2b4d22a41947e028b P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 1 FOLLOWER]: Refusing update from remote peer cd1618f314b441a99f3374fa93174aa1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:14:51.076366 18340 raft_consensus.cc:1261] T 94e30358b27845c2b4d22a41947e028b P 404270aa756047a288291fd808377592 [term 1 FOLLOWER]: Refusing update from remote peer cd1618f314b441a99f3374fa93174aa1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:14:51.076701 18724 consensus_queue.cc:1035] T 94e30358b27845c2b4d22a41947e028b P cd1618f314b441a99f3374fa93174aa1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:14:51.076843 18766 consensus_queue.cc:1035] T 94e30358b27845c2b4d22a41947e028b P cd1618f314b441a99f3374fa93174aa1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:14:51.085917 18713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:51.083513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:59837 (local address 127.15.217.67:34305)
0810 07:14:51.083555 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:51.083557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:51.083564 (+ 7us) server_negotiation.cc:407] Connection header received
0810 07:14:51.083583 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:51.083583 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:51.083591 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:51.083637 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:14:51.083784 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:51.084247 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:51.085320 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:51.085505 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:51.085554 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:51.085730 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:14:51.085755 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:14:51.085787 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:51.085845 (+ 58us) server_negotiation.cc:299] Negotiation successful
0810 07:14:51.085862 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 07:14:51.085975 18548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:51.083279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:34305 (local address 127.15.217.66:59837)
0810 07:14:51.083387 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:51.083399 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:51.083440 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:51.083661 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:51.083662 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:51.083664 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:14:51.083715 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:51.083717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:51.084322 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:51.084323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:51.085222 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:51.085226 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:51.085578 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:51.085580 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:51.085645 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:51.085651 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:14:51.085792 (+ 141us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:14:51.085793 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 07:14:51.085843 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 07:14:51.085887 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85}
W20260810 07:14:51.086541 18744 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 07:14:52.612725 18162 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:14:52.612834 18162 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:14:52.613696 18162 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1b66515d7fd64e6dbd7495ec1558830f: Sending DeleteTablet for 3 replicas of tablet 94e30358b27845c2b4d22a41947e028b
I20260810 07:14:52.614033 18619 tablet_service.cc:1514] Processing DeleteTablet for tablet 94e30358b27845c2b4d22a41947e028b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:14:52 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:37220
I20260810 07:14:52.614033 18317 tablet_service.cc:1514] Processing DeleteTablet for tablet 94e30358b27845c2b4d22a41947e028b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:14:52 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:34406
I20260810 07:14:52.614038 18467 tablet_service.cc:1514] Processing DeleteTablet for tablet 94e30358b27845c2b4d22a41947e028b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:14:52 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:58216
I20260810 07:14:52.614248 18797 tablet_replica.cc:330] T 94e30358b27845c2b4d22a41947e028b P f3e90ba48a6a44b5bb4dfbc06888f1c3: stopping tablet replica
I20260810 07:14:52.614308 18797 raft_consensus.cc:2229] T 94e30358b27845c2b4d22a41947e028b P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:14:52.614356 18797 raft_consensus.cc:2258] T 94e30358b27845c2b4d22a41947e028b P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:14:52.614780 18797 ts_tablet_manager.cc:1905] T 94e30358b27845c2b4d22a41947e028b P f3e90ba48a6a44b5bb4dfbc06888f1c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:14:52.616429 18797 ts_tablet_manager.cc:1918] T 94e30358b27845c2b4d22a41947e028b P f3e90ba48a6a44b5bb4dfbc06888f1c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 07:14:52.616549 18797 log.cc:1196] T 94e30358b27845c2b4d22a41947e028b P f3e90ba48a6a44b5bb4dfbc06888f1c3: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-1/wal/wals/94e30358b27845c2b4d22a41947e028b
I20260810 07:14:52.616788 18797 ts_tablet_manager.cc:1939] T 94e30358b27845c2b4d22a41947e028b P f3e90ba48a6a44b5bb4dfbc06888f1c3: Deleting consensus metadata
I20260810 07:14:52.617107 18148 catalog_manager.cc:4841] TS f3e90ba48a6a44b5bb4dfbc06888f1c3 (127.15.217.66:33449): tablet 94e30358b27845c2b4d22a41947e028b (table test-table [id=abe97e92904b49f5ba7a1a873c9546dc]) successfully deleted
I20260810 07:14:52.618664 18795 tablet_replica.cc:330] T 94e30358b27845c2b4d22a41947e028b P 404270aa756047a288291fd808377592: stopping tablet replica
I20260810 07:14:52.618700 18796 tablet_replica.cc:330] T 94e30358b27845c2b4d22a41947e028b P cd1618f314b441a99f3374fa93174aa1: stopping tablet replica
I20260810 07:14:52.618744 18795 raft_consensus.cc:2229] T 94e30358b27845c2b4d22a41947e028b P 404270aa756047a288291fd808377592 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:14:52.618779 18796 raft_consensus.cc:2229] T 94e30358b27845c2b4d22a41947e028b P cd1618f314b441a99f3374fa93174aa1 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:14:52.618817 18795 raft_consensus.cc:2258] T 94e30358b27845c2b4d22a41947e028b P 404270aa756047a288291fd808377592 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:14:52.618914 18796 raft_consensus.cc:2258] T 94e30358b27845c2b4d22a41947e028b P cd1618f314b441a99f3374fa93174aa1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:14:52.619091 18795 ts_tablet_manager.cc:1905] T 94e30358b27845c2b4d22a41947e028b P 404270aa756047a288291fd808377592: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:14:52.619181 18796 ts_tablet_manager.cc:1905] T 94e30358b27845c2b4d22a41947e028b P cd1618f314b441a99f3374fa93174aa1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:14:52.620565 18796 ts_tablet_manager.cc:1918] T 94e30358b27845c2b4d22a41947e028b P cd1618f314b441a99f3374fa93174aa1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 07:14:52.620632 18796 log.cc:1196] T 94e30358b27845c2b4d22a41947e028b P cd1618f314b441a99f3374fa93174aa1: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-2/wal/wals/94e30358b27845c2b4d22a41947e028b
I20260810 07:14:52.620908 18796 ts_tablet_manager.cc:1939] T 94e30358b27845c2b4d22a41947e028b P cd1618f314b441a99f3374fa93174aa1: Deleting consensus metadata
I20260810 07:14:52.621258 18147 catalog_manager.cc:4841] TS cd1618f314b441a99f3374fa93174aa1 (127.15.217.67:34305): tablet 94e30358b27845c2b4d22a41947e028b (table test-table [id=abe97e92904b49f5ba7a1a873c9546dc]) successfully deleted
I20260810 07:14:52.621598 18795 ts_tablet_manager.cc:1918] T 94e30358b27845c2b4d22a41947e028b P 404270aa756047a288291fd808377592: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 07:14:52.621678 18795 log.cc:1196] T 94e30358b27845c2b4d22a41947e028b P 404270aa756047a288291fd808377592: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-0/wal/wals/94e30358b27845c2b4d22a41947e028b
I20260810 07:14:52.621914 18795 ts_tablet_manager.cc:1939] T 94e30358b27845c2b4d22a41947e028b P 404270aa756047a288291fd808377592: Deleting consensus metadata
I20260810 07:14:52.622262 18148 catalog_manager.cc:4841] TS 404270aa756047a288291fd808377592 (127.15.217.65:34161): tablet 94e30358b27845c2b4d22a41947e028b (table test-table [id=abe97e92904b49f5ba7a1a873c9546dc]) successfully deleted
Aug 10 07:14:52 dist-test-slave-10pc krb5kdc[18115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346092, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 07:14:52 dist-test-slave-10pc krb5kdc[18115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346092, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:14:52.647109 18807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:52.634662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58930 (local address 127.15.217.126:41785)
0810 07:14:52.634811 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:52.634813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:52.634823 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:14:52.634873 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:52.634874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:52.634885 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:52.634946 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:52.635544 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:52.636130 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:52.636971 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:52.637190 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:52.637257 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:52.644395 (+ 7138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:52.644397 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:52.644397 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:52.644404 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:52.646038 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:52.646384 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:52.646385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:52.646385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:52.646442 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:52.646615 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:52.646615 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:52.646615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:52.646725 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:52.646824 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:52.646965 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 07:14:52.647033 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":93,"threads_started":1}
I20260810 07:14:52.648828 18162 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58930:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:14:52.648998 18162 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:14:52.650468 18317 tablet_service.cc:1467] Processing CreateTablet for tablet fdec0fc875aa486ba564654f262bcb6a (DEFAULT_TABLE table=test-table [id=a57c5f8a8aee4df683b9c234ae0e3633]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:14:52.650477 18467 tablet_service.cc:1467] Processing CreateTablet for tablet fdec0fc875aa486ba564654f262bcb6a (DEFAULT_TABLE table=test-table [id=a57c5f8a8aee4df683b9c234ae0e3633]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:14:52.650517 18619 tablet_service.cc:1467] Processing CreateTablet for tablet fdec0fc875aa486ba564654f262bcb6a (DEFAULT_TABLE table=test-table [id=a57c5f8a8aee4df683b9c234ae0e3633]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:14:52.650638 18619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdec0fc875aa486ba564654f262bcb6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:14:52.650640 18317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdec0fc875aa486ba564654f262bcb6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:14:52.650638 18467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdec0fc875aa486ba564654f262bcb6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:14:52.652309 18808 tablet_bootstrap.cc:492] T fdec0fc875aa486ba564654f262bcb6a P f3e90ba48a6a44b5bb4dfbc06888f1c3: Bootstrap starting.
I20260810 07:14:52.652524 18810 tablet_bootstrap.cc:492] T fdec0fc875aa486ba564654f262bcb6a P 404270aa756047a288291fd808377592: Bootstrap starting.
I20260810 07:14:52.652873 18808 tablet_bootstrap.cc:654] T fdec0fc875aa486ba564654f262bcb6a P f3e90ba48a6a44b5bb4dfbc06888f1c3: Neither blocks nor log segments found. Creating new log.
I20260810 07:14:52.652902 18810 tablet_bootstrap.cc:654] T fdec0fc875aa486ba564654f262bcb6a P 404270aa756047a288291fd808377592: Neither blocks nor log segments found. Creating new log.
I20260810 07:14:52.653228 18809 tablet_bootstrap.cc:492] T fdec0fc875aa486ba564654f262bcb6a P cd1618f314b441a99f3374fa93174aa1: Bootstrap starting.
I20260810 07:14:52.653697 18809 tablet_bootstrap.cc:654] T fdec0fc875aa486ba564654f262bcb6a P cd1618f314b441a99f3374fa93174aa1: Neither blocks nor log segments found. Creating new log.
I20260810 07:14:52.653702 18808 tablet_bootstrap.cc:492] T fdec0fc875aa486ba564654f262bcb6a P f3e90ba48a6a44b5bb4dfbc06888f1c3: No bootstrap required, opened a new log
I20260810 07:14:52.653864 18808 ts_tablet_manager.cc:1397] T fdec0fc875aa486ba564654f262bcb6a P f3e90ba48a6a44b5bb4dfbc06888f1c3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:14:52.653905 18810 tablet_bootstrap.cc:492] T fdec0fc875aa486ba564654f262bcb6a P 404270aa756047a288291fd808377592: No bootstrap required, opened a new log
I20260810 07:14:52.653959 18810 ts_tablet_manager.cc:1397] T fdec0fc875aa486ba564654f262bcb6a P 404270aa756047a288291fd808377592: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:14:52.654026 18808 raft_consensus.cc:348] T fdec0fc875aa486ba564654f262bcb6a P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } }
I20260810 07:14:52.654119 18808 raft_consensus.cc:374] T fdec0fc875aa486ba564654f262bcb6a P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:14:52.654140 18808 raft_consensus.cc:729] T fdec0fc875aa486ba564654f262bcb6a P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e90ba48a6a44b5bb4dfbc06888f1c3, State: Initialized, Role: FOLLOWER
I20260810 07:14:52.654104 18810 raft_consensus.cc:348] T fdec0fc875aa486ba564654f262bcb6a P 404270aa756047a288291fd808377592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } }
I20260810 07:14:52.654182 18810 raft_consensus.cc:374] T fdec0fc875aa486ba564654f262bcb6a P 404270aa756047a288291fd808377592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:14:52.654208 18810 raft_consensus.cc:729] T fdec0fc875aa486ba564654f262bcb6a P 404270aa756047a288291fd808377592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 404270aa756047a288291fd808377592, State: Initialized, Role: FOLLOWER
I20260810 07:14:52.654217 18808 consensus_queue.cc:260] T fdec0fc875aa486ba564654f262bcb6a P f3e90ba48a6a44b5bb4dfbc06888f1c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } }
I20260810 07:14:52.654273 18810 consensus_queue.cc:260] T fdec0fc875aa486ba564654f262bcb6a P 404270aa756047a288291fd808377592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } }
I20260810 07:14:52.654345 18808 ts_tablet_manager.cc:1428] T fdec0fc875aa486ba564654f262bcb6a P f3e90ba48a6a44b5bb4dfbc06888f1c3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:52.654480 18810 ts_tablet_manager.cc:1428] T fdec0fc875aa486ba564654f262bcb6a P 404270aa756047a288291fd808377592: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 07:14:52.654597 18809 tablet_bootstrap.cc:492] T fdec0fc875aa486ba564654f262bcb6a P cd1618f314b441a99f3374fa93174aa1: No bootstrap required, opened a new log
I20260810 07:14:52.654742 18809 ts_tablet_manager.cc:1397] T fdec0fc875aa486ba564654f262bcb6a P cd1618f314b441a99f3374fa93174aa1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:14:52.654863 18809 raft_consensus.cc:348] T fdec0fc875aa486ba564654f262bcb6a P cd1618f314b441a99f3374fa93174aa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } }
I20260810 07:14:52.654953 18809 raft_consensus.cc:374] T fdec0fc875aa486ba564654f262bcb6a P cd1618f314b441a99f3374fa93174aa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:14:52.654981 18809 raft_consensus.cc:729] T fdec0fc875aa486ba564654f262bcb6a P cd1618f314b441a99f3374fa93174aa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd1618f314b441a99f3374fa93174aa1, State: Initialized, Role: FOLLOWER
I20260810 07:14:52.655042 18809 consensus_queue.cc:260] T fdec0fc875aa486ba564654f262bcb6a P cd1618f314b441a99f3374fa93174aa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } }
I20260810 07:14:52.655203 18809 ts_tablet_manager.cc:1428] T fdec0fc875aa486ba564654f262bcb6a P cd1618f314b441a99f3374fa93174aa1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260810 07:14:52.742468 18694 tablet.cc:2369] T fdec0fc875aa486ba564654f262bcb6a P cd1618f314b441a99f3374fa93174aa1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:14:52.967185 18766 raft_consensus.cc:482] T fdec0fc875aa486ba564654f262bcb6a P cd1618f314b441a99f3374fa93174aa1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:14:52.967355 18766 raft_consensus.cc:504] T fdec0fc875aa486ba564654f262bcb6a P cd1618f314b441a99f3374fa93174aa1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } }
I20260810 07:14:52.967561 18766 leader_election.cc:288] T fdec0fc875aa486ba564654f262bcb6a P cd1618f314b441a99f3374fa93174aa1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 404270aa756047a288291fd808377592 (127.15.217.65:34161), f3e90ba48a6a44b5bb4dfbc06888f1c3 (127.15.217.66:33449)
I20260810 07:14:52.968014 18340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdec0fc875aa486ba564654f262bcb6a" candidate_uuid: "cd1618f314b441a99f3374fa93174aa1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "404270aa756047a288291fd808377592" is_pre_election: true
I20260810 07:14:52.968024 18490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdec0fc875aa486ba564654f262bcb6a" candidate_uuid: "cd1618f314b441a99f3374fa93174aa1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" is_pre_election: true
I20260810 07:14:52.968133 18490 raft_consensus.cc:2454] T fdec0fc875aa486ba564654f262bcb6a P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd1618f314b441a99f3374fa93174aa1 in term 0.
I20260810 07:14:52.968143 18340 raft_consensus.cc:2454] T fdec0fc875aa486ba564654f262bcb6a P 404270aa756047a288291fd808377592 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd1618f314b441a99f3374fa93174aa1 in term 0.
I20260810 07:14:52.968328 18571 leader_election.cc:302] T fdec0fc875aa486ba564654f262bcb6a P cd1618f314b441a99f3374fa93174aa1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 404270aa756047a288291fd808377592, cd1618f314b441a99f3374fa93174aa1; no voters:
I20260810 07:14:52.968441 18766 raft_consensus.cc:2781] T fdec0fc875aa486ba564654f262bcb6a P cd1618f314b441a99f3374fa93174aa1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:14:52.968511 18766 raft_consensus.cc:482] T fdec0fc875aa486ba564654f262bcb6a P cd1618f314b441a99f3374fa93174aa1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:14:52.968532 18766 raft_consensus.cc:3037] T fdec0fc875aa486ba564654f262bcb6a P cd1618f314b441a99f3374fa93174aa1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:14:52.969303 18766 raft_consensus.cc:504] T fdec0fc875aa486ba564654f262bcb6a P cd1618f314b441a99f3374fa93174aa1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } }
I20260810 07:14:52.969444 18766 leader_election.cc:288] T fdec0fc875aa486ba564654f262bcb6a P cd1618f314b441a99f3374fa93174aa1 [CANDIDATE]: Term 1 election: Requested vote from peers 404270aa756047a288291fd808377592 (127.15.217.65:34161), f3e90ba48a6a44b5bb4dfbc06888f1c3 (127.15.217.66:33449)
I20260810 07:14:52.969646 18340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdec0fc875aa486ba564654f262bcb6a" candidate_uuid: "cd1618f314b441a99f3374fa93174aa1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "404270aa756047a288291fd808377592"
I20260810 07:14:52.969659 18490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdec0fc875aa486ba564654f262bcb6a" candidate_uuid: "cd1618f314b441a99f3374fa93174aa1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3"
I20260810 07:14:52.969738 18340 raft_consensus.cc:3037] T fdec0fc875aa486ba564654f262bcb6a P 404270aa756047a288291fd808377592 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:14:52.969738 18490 raft_consensus.cc:3037] T fdec0fc875aa486ba564654f262bcb6a P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:14:52.970548 18340 raft_consensus.cc:2454] T fdec0fc875aa486ba564654f262bcb6a P 404270aa756047a288291fd808377592 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd1618f314b441a99f3374fa93174aa1 in term 1.
I20260810 07:14:52.970548 18490 raft_consensus.cc:2454] T fdec0fc875aa486ba564654f262bcb6a P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd1618f314b441a99f3374fa93174aa1 in term 1.
I20260810 07:14:52.970752 18571 leader_election.cc:302] T fdec0fc875aa486ba564654f262bcb6a P cd1618f314b441a99f3374fa93174aa1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 404270aa756047a288291fd808377592, cd1618f314b441a99f3374fa93174aa1; no voters:
I20260810 07:14:52.970865 18766 raft_consensus.cc:2781] T fdec0fc875aa486ba564654f262bcb6a P cd1618f314b441a99f3374fa93174aa1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:14:52.970924 18766 raft_consensus.cc:686] T fdec0fc875aa486ba564654f262bcb6a P cd1618f314b441a99f3374fa93174aa1 [term 1 LEADER]: Becoming Leader. State: Replica: cd1618f314b441a99f3374fa93174aa1, State: Running, Role: LEADER
I20260810 07:14:52.970979 18766 consensus_queue.cc:237] T fdec0fc875aa486ba564654f262bcb6a P cd1618f314b441a99f3374fa93174aa1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } }
I20260810 07:14:52.971711 18162 catalog_manager.cc:5495] T fdec0fc875aa486ba564654f262bcb6a P cd1618f314b441a99f3374fa93174aa1 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd1618f314b441a99f3374fa93174aa1 (127.15.217.67). New cstate: current_term: 1 leader_uuid: "cd1618f314b441a99f3374fa93174aa1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } health_report { overall_health: HEALTHY } } }
I20260810 07:14:53.011111 18820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:53.008562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37240 (local address 127.15.217.67:34305)
0810 07:14:53.008767 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:53.008769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:53.008777 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:14:53.008800 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:53.008800 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:53.008810 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:53.008888 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:14:53.009014 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:53.009503 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:53.010408 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:53.010606 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:53.010658 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:53.010849 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:14:53.010878 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:14:53.010917 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:53.011021 (+ 104us) server_negotiation.cc:299] Negotiation successful
0810 07:14:53.011042 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":114,"threads_started":1}
I20260810 07:14:53.013857 18340 raft_consensus.cc:1261] T fdec0fc875aa486ba564654f262bcb6a P 404270aa756047a288291fd808377592 [term 1 FOLLOWER]: Refusing update from remote peer cd1618f314b441a99f3374fa93174aa1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:14:53.013857 18490 raft_consensus.cc:1261] T fdec0fc875aa486ba564654f262bcb6a P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 1 FOLLOWER]: Refusing update from remote peer cd1618f314b441a99f3374fa93174aa1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:14:53.014127 18787 consensus_queue.cc:1035] T fdec0fc875aa486ba564654f262bcb6a P cd1618f314b441a99f3374fa93174aa1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:14:53.014274 18766 consensus_queue.cc:1035] T fdec0fc875aa486ba564654f262bcb6a P cd1618f314b441a99f3374fa93174aa1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:14:53.024504 18163 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:14:53.024637 18163 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:14:53.025327 18163 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1b66515d7fd64e6dbd7495ec1558830f: Sending DeleteTablet for 3 replicas of tablet fdec0fc875aa486ba564654f262bcb6a
I20260810 07:14:53.025588 18619 tablet_service.cc:1514] Processing DeleteTablet for tablet fdec0fc875aa486ba564654f262bcb6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:14:53 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:37220
I20260810 07:14:53.025632 18317 tablet_service.cc:1514] Processing DeleteTablet for tablet fdec0fc875aa486ba564654f262bcb6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:14:53 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:34406
I20260810 07:14:53.025650 18467 tablet_service.cc:1514] Processing DeleteTablet for tablet fdec0fc875aa486ba564654f262bcb6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:14:53 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:58216
I20260810 07:14:53.025686 18796 tablet_replica.cc:330] T fdec0fc875aa486ba564654f262bcb6a P cd1618f314b441a99f3374fa93174aa1: stopping tablet replica
I20260810 07:14:53.025735 18797 tablet_replica.cc:330] T fdec0fc875aa486ba564654f262bcb6a P f3e90ba48a6a44b5bb4dfbc06888f1c3: stopping tablet replica
I20260810 07:14:53.025738 18795 tablet_replica.cc:330] T fdec0fc875aa486ba564654f262bcb6a P 404270aa756047a288291fd808377592: stopping tablet replica
I20260810 07:14:53.025745 18796 raft_consensus.cc:2229] T fdec0fc875aa486ba564654f262bcb6a P cd1618f314b441a99f3374fa93174aa1 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:14:53.025785 18795 raft_consensus.cc:2229] T fdec0fc875aa486ba564654f262bcb6a P 404270aa756047a288291fd808377592 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:14:53.025799 18796 raft_consensus.cc:2258] T fdec0fc875aa486ba564654f262bcb6a P cd1618f314b441a99f3374fa93174aa1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:14:53.025828 18795 raft_consensus.cc:2258] T fdec0fc875aa486ba564654f262bcb6a P 404270aa756047a288291fd808377592 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:14:53.025785 18797 raft_consensus.cc:2229] T fdec0fc875aa486ba564654f262bcb6a P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:14:53.025869 18797 raft_consensus.cc:2258] T fdec0fc875aa486ba564654f262bcb6a P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:14:53.026134 18796 ts_tablet_manager.cc:1905] T fdec0fc875aa486ba564654f262bcb6a P cd1618f314b441a99f3374fa93174aa1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:14:53.026250 18795 ts_tablet_manager.cc:1905] T fdec0fc875aa486ba564654f262bcb6a P 404270aa756047a288291fd808377592: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:14:53.026294 18163 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58930:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:14:53.026433 18163 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:14:53.027726 18619 tablet_service.cc:1467] Processing CreateTablet for tablet 17a689b438924b9a8f14f8e22e7efbc0 (DEFAULT_TABLE table=test-table [id=dcca6b33d80f4cf09f13db2fa38bf5f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:14:53.027734 18317 tablet_service.cc:1467] Processing CreateTablet for tablet 17a689b438924b9a8f14f8e22e7efbc0 (DEFAULT_TABLE table=test-table [id=dcca6b33d80f4cf09f13db2fa38bf5f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:14:53.027846 18619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17a689b438924b9a8f14f8e22e7efbc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:14:53.027948 18796 ts_tablet_manager.cc:1918] T fdec0fc875aa486ba564654f262bcb6a P cd1618f314b441a99f3374fa93174aa1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:14:53.027942 18795 ts_tablet_manager.cc:1918] T fdec0fc875aa486ba564654f262bcb6a P 404270aa756047a288291fd808377592: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:14:53.027995 18796 log.cc:1196] T fdec0fc875aa486ba564654f262bcb6a P cd1618f314b441a99f3374fa93174aa1: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-2/wal/wals/fdec0fc875aa486ba564654f262bcb6a
I20260810 07:14:53.027945 18317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17a689b438924b9a8f14f8e22e7efbc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:14:53.028249 18796 ts_tablet_manager.cc:1939] T fdec0fc875aa486ba564654f262bcb6a P cd1618f314b441a99f3374fa93174aa1: Deleting consensus metadata
I20260810 07:14:53.028648 18797 ts_tablet_manager.cc:1905] T fdec0fc875aa486ba564654f262bcb6a P f3e90ba48a6a44b5bb4dfbc06888f1c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:14:53.029043 18810 tablet_bootstrap.cc:492] T 17a689b438924b9a8f14f8e22e7efbc0 P 404270aa756047a288291fd808377592: Bootstrap starting.
I20260810 07:14:53.029280 18147 catalog_manager.cc:4841] TS cd1618f314b441a99f3374fa93174aa1 (127.15.217.67:34305): tablet fdec0fc875aa486ba564654f262bcb6a (table test-table [id=a57c5f8a8aee4df683b9c234ae0e3633]) successfully deleted
I20260810 07:14:53.029557 18467 tablet_service.cc:1467] Processing CreateTablet for tablet 17a689b438924b9a8f14f8e22e7efbc0 (DEFAULT_TABLE table=test-table [id=dcca6b33d80f4cf09f13db2fa38bf5f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:14:53.029703 18467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17a689b438924b9a8f14f8e22e7efbc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:14:53.030340 18797 ts_tablet_manager.cc:1918] T fdec0fc875aa486ba564654f262bcb6a P f3e90ba48a6a44b5bb4dfbc06888f1c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:14:53.030435 18797 log.cc:1196] T fdec0fc875aa486ba564654f262bcb6a P f3e90ba48a6a44b5bb4dfbc06888f1c3: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-1/wal/wals/fdec0fc875aa486ba564654f262bcb6a
I20260810 07:14:53.030440 18810 tablet_bootstrap.cc:654] T 17a689b438924b9a8f14f8e22e7efbc0 P 404270aa756047a288291fd808377592: Neither blocks nor log segments found. Creating new log.
I20260810 07:14:53.030663 18797 ts_tablet_manager.cc:1939] T fdec0fc875aa486ba564654f262bcb6a P f3e90ba48a6a44b5bb4dfbc06888f1c3: Deleting consensus metadata
I20260810 07:14:53.030961 18809 tablet_bootstrap.cc:492] T 17a689b438924b9a8f14f8e22e7efbc0 P cd1618f314b441a99f3374fa93174aa1: Bootstrap starting.
I20260810 07:14:53.031018 18148 catalog_manager.cc:4841] TS f3e90ba48a6a44b5bb4dfbc06888f1c3 (127.15.217.66:33449): tablet fdec0fc875aa486ba564654f262bcb6a (table test-table [id=a57c5f8a8aee4df683b9c234ae0e3633]) successfully deleted
I20260810 07:14:53.031308 18809 tablet_bootstrap.cc:654] T 17a689b438924b9a8f14f8e22e7efbc0 P cd1618f314b441a99f3374fa93174aa1: Neither blocks nor log segments found. Creating new log.
I20260810 07:14:53.031585 18808 tablet_bootstrap.cc:492] T 17a689b438924b9a8f14f8e22e7efbc0 P f3e90ba48a6a44b5bb4dfbc06888f1c3: Bootstrap starting.
I20260810 07:14:53.031678 18810 tablet_bootstrap.cc:492] T 17a689b438924b9a8f14f8e22e7efbc0 P 404270aa756047a288291fd808377592: No bootstrap required, opened a new log
I20260810 07:14:53.028002 18795 log.cc:1196] T fdec0fc875aa486ba564654f262bcb6a P 404270aa756047a288291fd808377592: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-0/wal/wals/fdec0fc875aa486ba564654f262bcb6a
I20260810 07:14:53.031736 18810 ts_tablet_manager.cc:1397] T 17a689b438924b9a8f14f8e22e7efbc0 P 404270aa756047a288291fd808377592: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:14:53.031904 18808 tablet_bootstrap.cc:654] T 17a689b438924b9a8f14f8e22e7efbc0 P f3e90ba48a6a44b5bb4dfbc06888f1c3: Neither blocks nor log segments found. Creating new log.
I20260810 07:14:53.031960 18809 tablet_bootstrap.cc:492] T 17a689b438924b9a8f14f8e22e7efbc0 P cd1618f314b441a99f3374fa93174aa1: No bootstrap required, opened a new log
I20260810 07:14:53.032003 18809 ts_tablet_manager.cc:1397] T 17a689b438924b9a8f14f8e22e7efbc0 P cd1618f314b441a99f3374fa93174aa1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:14:53.032066 18795 ts_tablet_manager.cc:1939] T fdec0fc875aa486ba564654f262bcb6a P 404270aa756047a288291fd808377592: Deleting consensus metadata
I20260810 07:14:53.032135 18809 raft_consensus.cc:348] T 17a689b438924b9a8f14f8e22e7efbc0 P cd1618f314b441a99f3374fa93174aa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } }
I20260810 07:14:53.032137 18810 raft_consensus.cc:348] T 17a689b438924b9a8f14f8e22e7efbc0 P 404270aa756047a288291fd808377592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } }
I20260810 07:14:53.032207 18809 raft_consensus.cc:374] T 17a689b438924b9a8f14f8e22e7efbc0 P cd1618f314b441a99f3374fa93174aa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:14:53.032222 18809 raft_consensus.cc:729] T 17a689b438924b9a8f14f8e22e7efbc0 P cd1618f314b441a99f3374fa93174aa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd1618f314b441a99f3374fa93174aa1, State: Initialized, Role: FOLLOWER
I20260810 07:14:53.032217 18810 raft_consensus.cc:374] T 17a689b438924b9a8f14f8e22e7efbc0 P 404270aa756047a288291fd808377592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:14:53.032243 18810 raft_consensus.cc:729] T 17a689b438924b9a8f14f8e22e7efbc0 P 404270aa756047a288291fd808377592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 404270aa756047a288291fd808377592, State: Initialized, Role: FOLLOWER
I20260810 07:14:53.032282 18809 consensus_queue.cc:260] T 17a689b438924b9a8f14f8e22e7efbc0 P cd1618f314b441a99f3374fa93174aa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } }
I20260810 07:14:53.032287 18810 consensus_queue.cc:260] T 17a689b438924b9a8f14f8e22e7efbc0 P 404270aa756047a288291fd808377592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } }
I20260810 07:14:53.032388 18809 ts_tablet_manager.cc:1428] T 17a689b438924b9a8f14f8e22e7efbc0 P cd1618f314b441a99f3374fa93174aa1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:53.032421 18148 catalog_manager.cc:4841] TS 404270aa756047a288291fd808377592 (127.15.217.65:34161): tablet fdec0fc875aa486ba564654f262bcb6a (table test-table [id=a57c5f8a8aee4df683b9c234ae0e3633]) successfully deleted
I20260810 07:14:53.032825 18808 tablet_bootstrap.cc:492] T 17a689b438924b9a8f14f8e22e7efbc0 P f3e90ba48a6a44b5bb4dfbc06888f1c3: No bootstrap required, opened a new log
I20260810 07:14:53.032881 18808 ts_tablet_manager.cc:1397] T 17a689b438924b9a8f14f8e22e7efbc0 P f3e90ba48a6a44b5bb4dfbc06888f1c3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:14:53.032822 18810 ts_tablet_manager.cc:1428] T 17a689b438924b9a8f14f8e22e7efbc0 P 404270aa756047a288291fd808377592: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:14:53.033082 18808 raft_consensus.cc:348] T 17a689b438924b9a8f14f8e22e7efbc0 P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } }
I20260810 07:14:53.033182 18808 raft_consensus.cc:374] T 17a689b438924b9a8f14f8e22e7efbc0 P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:14:53.033215 18808 raft_consensus.cc:729] T 17a689b438924b9a8f14f8e22e7efbc0 P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e90ba48a6a44b5bb4dfbc06888f1c3, State: Initialized, Role: FOLLOWER
I20260810 07:14:53.033262 18808 consensus_queue.cc:260] T 17a689b438924b9a8f14f8e22e7efbc0 P f3e90ba48a6a44b5bb4dfbc06888f1c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } }
I20260810 07:14:53.033357 18808 ts_tablet_manager.cc:1428] T 17a689b438924b9a8f14f8e22e7efbc0 P f3e90ba48a6a44b5bb4dfbc06888f1c3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:53.153685 18766 raft_consensus.cc:482] T 17a689b438924b9a8f14f8e22e7efbc0 P cd1618f314b441a99f3374fa93174aa1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:14:53.153848 18766 raft_consensus.cc:504] T 17a689b438924b9a8f14f8e22e7efbc0 P cd1618f314b441a99f3374fa93174aa1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } }
I20260810 07:14:53.154107 18766 leader_election.cc:288] T 17a689b438924b9a8f14f8e22e7efbc0 P cd1618f314b441a99f3374fa93174aa1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 404270aa756047a288291fd808377592 (127.15.217.65:34161), f3e90ba48a6a44b5bb4dfbc06888f1c3 (127.15.217.66:33449)
I20260810 07:14:53.154451 18339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17a689b438924b9a8f14f8e22e7efbc0" candidate_uuid: "cd1618f314b441a99f3374fa93174aa1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "404270aa756047a288291fd808377592" is_pre_election: true
I20260810 07:14:53.154456 18490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17a689b438924b9a8f14f8e22e7efbc0" candidate_uuid: "cd1618f314b441a99f3374fa93174aa1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" is_pre_election: true
I20260810 07:14:53.154559 18339 raft_consensus.cc:2454] T 17a689b438924b9a8f14f8e22e7efbc0 P 404270aa756047a288291fd808377592 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd1618f314b441a99f3374fa93174aa1 in term 0.
I20260810 07:14:53.154579 18490 raft_consensus.cc:2454] T 17a689b438924b9a8f14f8e22e7efbc0 P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd1618f314b441a99f3374fa93174aa1 in term 0.
I20260810 07:14:53.154757 18571 leader_election.cc:302] T 17a689b438924b9a8f14f8e22e7efbc0 P cd1618f314b441a99f3374fa93174aa1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd1618f314b441a99f3374fa93174aa1, f3e90ba48a6a44b5bb4dfbc06888f1c3; no voters:
I20260810 07:14:53.154903 18766 raft_consensus.cc:2781] T 17a689b438924b9a8f14f8e22e7efbc0 P cd1618f314b441a99f3374fa93174aa1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:14:53.154964 18766 raft_consensus.cc:482] T 17a689b438924b9a8f14f8e22e7efbc0 P cd1618f314b441a99f3374fa93174aa1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:14:53.154980 18766 raft_consensus.cc:3037] T 17a689b438924b9a8f14f8e22e7efbc0 P cd1618f314b441a99f3374fa93174aa1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:14:53.155728 18766 raft_consensus.cc:504] T 17a689b438924b9a8f14f8e22e7efbc0 P cd1618f314b441a99f3374fa93174aa1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } }
I20260810 07:14:53.155848 18766 leader_election.cc:288] T 17a689b438924b9a8f14f8e22e7efbc0 P cd1618f314b441a99f3374fa93174aa1 [CANDIDATE]: Term 1 election: Requested vote from peers 404270aa756047a288291fd808377592 (127.15.217.65:34161), f3e90ba48a6a44b5bb4dfbc06888f1c3 (127.15.217.66:33449)
I20260810 07:14:53.156121 18339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17a689b438924b9a8f14f8e22e7efbc0" candidate_uuid: "cd1618f314b441a99f3374fa93174aa1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "404270aa756047a288291fd808377592"
I20260810 07:14:53.156126 18490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17a689b438924b9a8f14f8e22e7efbc0" candidate_uuid: "cd1618f314b441a99f3374fa93174aa1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3"
I20260810 07:14:53.156206 18339 raft_consensus.cc:3037] T 17a689b438924b9a8f14f8e22e7efbc0 P 404270aa756047a288291fd808377592 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:14:53.156232 18490 raft_consensus.cc:3037] T 17a689b438924b9a8f14f8e22e7efbc0 P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:14:53.157033 18490 raft_consensus.cc:2454] T 17a689b438924b9a8f14f8e22e7efbc0 P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd1618f314b441a99f3374fa93174aa1 in term 1.
I20260810 07:14:53.157272 18571 leader_election.cc:302] T 17a689b438924b9a8f14f8e22e7efbc0 P cd1618f314b441a99f3374fa93174aa1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd1618f314b441a99f3374fa93174aa1, f3e90ba48a6a44b5bb4dfbc06888f1c3; no voters:
I20260810 07:14:53.157397 18766 raft_consensus.cc:2781] T 17a689b438924b9a8f14f8e22e7efbc0 P cd1618f314b441a99f3374fa93174aa1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:14:53.157393 18339 raft_consensus.cc:2454] T 17a689b438924b9a8f14f8e22e7efbc0 P 404270aa756047a288291fd808377592 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd1618f314b441a99f3374fa93174aa1 in term 1.
I20260810 07:14:53.157477 18766 raft_consensus.cc:686] T 17a689b438924b9a8f14f8e22e7efbc0 P cd1618f314b441a99f3374fa93174aa1 [term 1 LEADER]: Becoming Leader. State: Replica: cd1618f314b441a99f3374fa93174aa1, State: Running, Role: LEADER
I20260810 07:14:53.157548 18766 consensus_queue.cc:237] T 17a689b438924b9a8f14f8e22e7efbc0 P cd1618f314b441a99f3374fa93174aa1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } }
I20260810 07:14:53.158257 18162 catalog_manager.cc:5495] T 17a689b438924b9a8f14f8e22e7efbc0 P cd1618f314b441a99f3374fa93174aa1 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd1618f314b441a99f3374fa93174aa1 (127.15.217.67). New cstate: current_term: 1 leader_uuid: "cd1618f314b441a99f3374fa93174aa1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd1618f314b441a99f3374fa93174aa1" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 34305 } health_report { overall_health: HEALTHY } } }
I20260810 07:14:53.183025 18339 raft_consensus.cc:1261] T 17a689b438924b9a8f14f8e22e7efbc0 P 404270aa756047a288291fd808377592 [term 1 FOLLOWER]: Refusing update from remote peer cd1618f314b441a99f3374fa93174aa1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:14:53.183285 18766 consensus_queue.cc:1035] T 17a689b438924b9a8f14f8e22e7efbc0 P cd1618f314b441a99f3374fa93174aa1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "404270aa756047a288291fd808377592" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:14:53.183359 18490 raft_consensus.cc:1261] T 17a689b438924b9a8f14f8e22e7efbc0 P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 1 FOLLOWER]: Refusing update from remote peer cd1618f314b441a99f3374fa93174aa1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:14:53.183522 18766 consensus_queue.cc:1035] T 17a689b438924b9a8f14f8e22e7efbc0 P cd1618f314b441a99f3374fa93174aa1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e90ba48a6a44b5bb4dfbc06888f1c3" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 07:14:53.188571 18777 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 07:14:54.706692 18163 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:14:54.706810 18163 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:14:54.707597 18163 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1b66515d7fd64e6dbd7495ec1558830f: Sending DeleteTablet for 3 replicas of tablet 17a689b438924b9a8f14f8e22e7efbc0
I20260810 07:14:54.707866 18317 tablet_service.cc:1514] Processing DeleteTablet for tablet 17a689b438924b9a8f14f8e22e7efbc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:14:54 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:34406
I20260810 07:14:54.707888 18468 tablet_service.cc:1514] Processing DeleteTablet for tablet 17a689b438924b9a8f14f8e22e7efbc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:14:54 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:58216
I20260810 07:14:54.708024 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 18242
I20260810 07:14:54.708073 18619 tablet_service.cc:1514] Processing DeleteTablet for tablet 17a689b438924b9a8f14f8e22e7efbc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:14:54 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:37220
I20260810 07:14:54.710886 18858 tablet_replica.cc:330] T 17a689b438924b9a8f14f8e22e7efbc0 P cd1618f314b441a99f3374fa93174aa1: stopping tablet replica
I20260810 07:14:54.710975 18858 raft_consensus.cc:2229] T 17a689b438924b9a8f14f8e22e7efbc0 P cd1618f314b441a99f3374fa93174aa1 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:14:54.711050 18858 raft_consensus.cc:2258] T 17a689b438924b9a8f14f8e22e7efbc0 P cd1618f314b441a99f3374fa93174aa1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:14:54.711293 18858 ts_tablet_manager.cc:1905] T 17a689b438924b9a8f14f8e22e7efbc0 P cd1618f314b441a99f3374fa93174aa1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:14:54.713250 18857 tablet_replica.cc:330] T 17a689b438924b9a8f14f8e22e7efbc0 P f3e90ba48a6a44b5bb4dfbc06888f1c3: stopping tablet replica
I20260810 07:14:54.713320 18857 raft_consensus.cc:2229] T 17a689b438924b9a8f14f8e22e7efbc0 P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:14:54.713377 18857 raft_consensus.cc:2258] T 17a689b438924b9a8f14f8e22e7efbc0 P f3e90ba48a6a44b5bb4dfbc06888f1c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:14:54.713551 18858 ts_tablet_manager.cc:1918] T 17a689b438924b9a8f14f8e22e7efbc0 P cd1618f314b441a99f3374fa93174aa1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 07:14:54.713619 18858 log.cc:1196] T 17a689b438924b9a8f14f8e22e7efbc0 P cd1618f314b441a99f3374fa93174aa1: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-2/wal/wals/17a689b438924b9a8f14f8e22e7efbc0
I20260810 07:14:54.713670 18857 ts_tablet_manager.cc:1905] T 17a689b438924b9a8f14f8e22e7efbc0 P f3e90ba48a6a44b5bb4dfbc06888f1c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:14:54.713899 18858 ts_tablet_manager.cc:1939] T 17a689b438924b9a8f14f8e22e7efbc0 P cd1618f314b441a99f3374fa93174aa1: Deleting consensus metadata
I20260810 07:14:54.714263 18147 catalog_manager.cc:4841] TS cd1618f314b441a99f3374fa93174aa1 (127.15.217.67:34305): tablet 17a689b438924b9a8f14f8e22e7efbc0 (table test-table [id=dcca6b33d80f4cf09f13db2fa38bf5f8]) successfully deleted
I20260810 07:14:54.715154 18857 ts_tablet_manager.cc:1918] T 17a689b438924b9a8f14f8e22e7efbc0 P f3e90ba48a6a44b5bb4dfbc06888f1c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 07:14:54.715212 18857 log.cc:1196] T 17a689b438924b9a8f14f8e22e7efbc0 P f3e90ba48a6a44b5bb4dfbc06888f1c3: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346081759552-16229-0/minicluster-data/ts-1/wal/wals/17a689b438924b9a8f14f8e22e7efbc0
I20260810 07:14:54.715440 18857 ts_tablet_manager.cc:1939] T 17a689b438924b9a8f14f8e22e7efbc0 P f3e90ba48a6a44b5bb4dfbc06888f1c3: Deleting consensus metadata
I20260810 07:14:54.715816 18148 catalog_manager.cc:4841] TS f3e90ba48a6a44b5bb4dfbc06888f1c3 (127.15.217.66:33449): tablet 17a689b438924b9a8f14f8e22e7efbc0 (table test-table [id=dcca6b33d80f4cf09f13db2fa38bf5f8]) successfully deleted
W20260810 07:14:54.718801 18148 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.217.65:34161 (error 108)
I20260810 07:14:54.719079 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 18398
I20260810 07:14:54.723778 18860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:54.719219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:34161 (local address 127.0.0.1:34416)
0810 07:14:54.723589 (+ 4370us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:54.723704 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.65:34161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4320,"thread_start_us":62,"threads_started":1}
W20260810 07:14:54.724009 18148 catalog_manager.cc:4584] TS 404270aa756047a288291fd808377592 (127.15.217.65:34161): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 17a689b438924b9a8f14f8e22e7efbc0: Network error: Client connection negotiation failed: client connection to 127.15.217.65:34161: connect: Connection refused (error 111)
I20260810 07:14:54.733700 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 18551
I20260810 07:14:54.744441 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 18129
2026-08-10T07:14:54Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5929 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:14:56 dist-test-slave-10pc krb5kdc[18863](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:14:56 dist-test-slave-10pc krb5kdc[18863](info): set up 2 sockets
Aug 10 07:14:56 dist-test-slave-10pc krb5kdc[18863](info): commencing operation
krb5kdc: starting...
W20260810 07:14:57.095819 16229 mini_kdc.cc:121] Time spent starting KDC: real 2.326s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:14:57 dist-test-slave-10pc krb5kdc[18863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:14:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:14:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
I20260810 07:14:57.228300 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:35641
--webserver_interface=127.15.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:38807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:35641
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:14:57.315315 18877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:14:57.315589 18877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:14:57.315634 18877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:14:57.316902 18877 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:14:57.316980 18877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:14:57.317005 18877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:14:57.317025 18877 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:14:57.317054 18877 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:14:57.318631 18877 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:38807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:35641
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:35641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:14:57.318939 18877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:14:57.319204 18877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:14:57.324937 18884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:57.324946 18883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:14:57.325184 18877 server_base.cc:1034] running on GCE node
W20260810 07:14:57.324946 18886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:14:57.325484 18877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:14:57.325809 18877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:14:57.327013 18877 hybrid_clock.cc:648] HybridClock initialized: now 1786346097326985 us; error 55 us; skew 500 ppm
Aug 10 07:14:57 dist-test-slave-10pc krb5kdc[18863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346097, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:14:57.329778 18877 init.cc:377] Logged in from keytab as kudu/127.15.217.126@KRBTEST.COM (short username kudu)
I20260810 07:14:57.330273 18877 webserver.cc:460] Webserver started at http://127.15.217.126:39325/ using document root <none> and password file <none>
I20260810 07:14:57.330549 18877 fs_manager.cc:359] Metadata directory not provided
I20260810 07:14:57.330619 18877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:14:57.330775 18877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:14:57.331900 18877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "2b0bcb74fab44f48882f862e6bc04b78"
format_stamp: "Formatted at 2026-08-10 07:14:57 on dist-test-slave-10pc"
server_key: "6bf455be47b8e60e60b185b1f1046ad6"
server_key_iv: "ad7a6328ac979493d1ecdfe67f535727"
server_key_version: "encryptionkey@0"
I20260810 07:14:57.332296 18877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "2b0bcb74fab44f48882f862e6bc04b78"
format_stamp: "Formatted at 2026-08-10 07:14:57 on dist-test-slave-10pc"
server_key: "6bf455be47b8e60e60b185b1f1046ad6"
server_key_iv: "ad7a6328ac979493d1ecdfe67f535727"
server_key_version: "encryptionkey@0"
I20260810 07:14:57.333765 18877 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:14:57.334576 18893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:57.334748 18877 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:57.334853 18877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "2b0bcb74fab44f48882f862e6bc04b78"
format_stamp: "Formatted at 2026-08-10 07:14:57 on dist-test-slave-10pc"
server_key: "6bf455be47b8e60e60b185b1f1046ad6"
server_key_iv: "ad7a6328ac979493d1ecdfe67f535727"
server_key_version: "encryptionkey@0"
I20260810 07:14:57.334929 18877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:14:57.345414 18877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:14:57.347776 18877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:14:57.347947 18877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:14:57.352229 18877 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:35641
I20260810 07:14:57.352267 18950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:35641 every 8 connection(s)
I20260810 07:14:57.352731 18877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:14:57.353647 18951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:14:57.354390 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 18877
I20260810 07:14:57.354493 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:14:57.354686 16229 external_mini_cluster.cc:1428] Setting key 41de7f946d92cc244a9baf9bdb2e40fc
I20260810 07:14:57.356956 18951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b0bcb74fab44f48882f862e6bc04b78: Bootstrap starting.
I20260810 07:14:57.357617 18951 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b0bcb74fab44f48882f862e6bc04b78: Neither blocks nor log segments found. Creating new log.
I20260810 07:14:57.357916 18951 log.cc:824] T 00000000000000000000000000000000 P 2b0bcb74fab44f48882f862e6bc04b78: Log is configured to *not* fsync() on all Append() calls
I20260810 07:14:57.358712 18951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b0bcb74fab44f48882f862e6bc04b78: No bootstrap required, opened a new log
I20260810 07:14:57.361477 18951 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b0bcb74fab44f48882f862e6bc04b78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b0bcb74fab44f48882f862e6bc04b78" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35641 } }
I20260810 07:14:57.361624 18951 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b0bcb74fab44f48882f862e6bc04b78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:14:57.361646 18951 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b0bcb74fab44f48882f862e6bc04b78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b0bcb74fab44f48882f862e6bc04b78, State: Initialized, Role: FOLLOWER
I20260810 07:14:57.361785 18951 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b0bcb74fab44f48882f862e6bc04b78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b0bcb74fab44f48882f862e6bc04b78" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35641 } }
I20260810 07:14:57.361861 18951 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b0bcb74fab44f48882f862e6bc04b78 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
Aug 10 07:14:57 dist-test-slave-10pc krb5kdc[18863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:14:57.361884 18951 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b0bcb74fab44f48882f862e6bc04b78 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:14:57.361948 18951 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b0bcb74fab44f48882f862e6bc04b78 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:14:57.362593 18951 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b0bcb74fab44f48882f862e6bc04b78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b0bcb74fab44f48882f862e6bc04b78" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35641 } }
I20260810 07:14:57.362725 18951 leader_election.cc:302] T 00000000000000000000000000000000 P 2b0bcb74fab44f48882f862e6bc04b78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b0bcb74fab44f48882f862e6bc04b78; no voters:
I20260810 07:14:57.362914 18951 leader_election.cc:288] T 00000000000000000000000000000000 P 2b0bcb74fab44f48882f862e6bc04b78 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:14:57.363068 18956 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b0bcb74fab44f48882f862e6bc04b78 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:14:57.363209 18951 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b0bcb74fab44f48882f862e6bc04b78 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:14:57.363271 18956 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b0bcb74fab44f48882f862e6bc04b78 [term 1 LEADER]: Becoming Leader. State: Replica: 2b0bcb74fab44f48882f862e6bc04b78, State: Running, Role: LEADER
I20260810 07:14:57.363734 18956 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b0bcb74fab44f48882f862e6bc04b78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b0bcb74fab44f48882f862e6bc04b78" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35641 } }
I20260810 07:14:57.364372 18957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b0bcb74fab44f48882f862e6bc04b78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b0bcb74fab44f48882f862e6bc04b78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b0bcb74fab44f48882f862e6bc04b78" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35641 } } }
I20260810 07:14:57.364480 18957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b0bcb74fab44f48882f862e6bc04b78 [sys.catalog]: This master's current role is: LEADER
I20260810 07:14:57.364745 18958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b0bcb74fab44f48882f862e6bc04b78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b0bcb74fab44f48882f862e6bc04b78. Latest consensus state: current_term: 1 leader_uuid: "2b0bcb74fab44f48882f862e6bc04b78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b0bcb74fab44f48882f862e6bc04b78" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35641 } } }
W20260810 07:14:57.365041 18973 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2b0bcb74fab44f48882f862e6bc04b78: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260810 07:14:57.365037 18958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b0bcb74fab44f48882f862e6bc04b78 [sys.catalog]: This master's current role is: LEADER
W20260810 07:14:57.365131 18973 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:14:57.366523 18974 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:14:57.366597 18954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:57.355226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47938 (local address 127.15.217.126:35641)
0810 07:14:57.355406 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:57.355414 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:57.355440 (+ 26us) server_negotiation.cc:407] Connection header received
0810 07:14:57.355885 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:57.355893 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:57.355940 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:57.356056 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:57.356935 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:57.357942 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:57.358998 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:57.359419 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:57.359539 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:57.362398 (+ 2859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:57.362403 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:57.362403 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:57.362418 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:57.364875 (+ 2457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:57.365339 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:57.365342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:57.365342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:57.365454 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:57.365662 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:57.365669 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:57.365669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:57.365980 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:57.366118 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:57.366236 (+ 118us) server_negotiation.cc:299] Negotiation successful
0810 07:14:57.366384 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":91,"threads_started":1}
I20260810 07:14:57.366997 18974 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:14:57.368570 18974 catalog_manager.cc:1347] Generated new cluster ID: 2d2dbbfe6a8743a9a5d62b6aeb6708b3
I20260810 07:14:57.368628 18974 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:14:57.401787 18974 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:14:57.402442 18974 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:14:57.407347 18974 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b0bcb74fab44f48882f862e6bc04b78: Generated new TSK 0
I20260810 07:14:57.407549 18974 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab.
Entry for principal kudu/127.15.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab.
I20260810 07:14:57.443066 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.65:0
--local_ip_for_outbound_sockets=127.15.217.65
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:35641
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:38807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:14:57.530551 18982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:14:57.530797 18982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:14:57.530839 18982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:14:57.532037 18982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:14:57.532107 18982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:14:57.532152 18982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.65
I20260810 07:14:57.533690 18982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:38807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:35641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.65
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:14:57.533974 18982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:14:57.534226 18982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:14:57.539523 18988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:57.539517 18990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:57.539522 18987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:14:57.540163 18982 server_base.cc:1034] running on GCE node
I20260810 07:14:57.540361 18982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:14:57.540627 18982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:14:57.541782 18982 hybrid_clock.cc:648] HybridClock initialized: now 1786346097541767 us; error 29 us; skew 500 ppm
Aug 10 07:14:57 dist-test-slave-10pc krb5kdc[18863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346097, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:14:57.544430 18982 init.cc:377] Logged in from keytab as kudu/127.15.217.65@KRBTEST.COM (short username kudu)
I20260810 07:14:57.544955 18982 webserver.cc:460] Webserver started at http://127.15.217.65:42895/ using document root <none> and password file <none>
I20260810 07:14:57.545171 18982 fs_manager.cc:359] Metadata directory not provided
I20260810 07:14:57.545215 18982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:14:57.545311 18982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:14:57.546399 18982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-0/data/instance:
uuid: "c32244d835a54308bbd761426ae88c3c"
format_stamp: "Formatted at 2026-08-10 07:14:57 on dist-test-slave-10pc"
server_key: "5f01cc2ccfe0a1fc8891a23a8dad20fc"
server_key_iv: "6e5ca25f306b9e4ee9561c49cb3b9f1b"
server_key_version: "encryptionkey@0"
I20260810 07:14:57.546753 18982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance:
uuid: "c32244d835a54308bbd761426ae88c3c"
format_stamp: "Formatted at 2026-08-10 07:14:57 on dist-test-slave-10pc"
server_key: "5f01cc2ccfe0a1fc8891a23a8dad20fc"
server_key_iv: "6e5ca25f306b9e4ee9561c49cb3b9f1b"
server_key_version: "encryptionkey@0"
I20260810 07:14:57.548184 18982 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:14:57.549393 18998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:57.549584 18982 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:14:57.549682 18982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-0/wal
uuid: "c32244d835a54308bbd761426ae88c3c"
format_stamp: "Formatted at 2026-08-10 07:14:57 on dist-test-slave-10pc"
server_key: "5f01cc2ccfe0a1fc8891a23a8dad20fc"
server_key_iv: "6e5ca25f306b9e4ee9561c49cb3b9f1b"
server_key_version: "encryptionkey@0"
I20260810 07:14:57.549744 18982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:14:57.571274 18982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:14:57.573381 18982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:14:57.573505 18982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:14:57.573711 18982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:14:57.574052 18982 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:14:57.574102 18982 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:57.574157 18982 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:14:57.574186 18982 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:57.580317 18982 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.65:44479
I20260810 07:14:57.580411 19124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.65:44479 every 8 connection(s)
I20260810 07:14:57.580782 18982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
Aug 10 07:14:57 dist-test-slave-10pc krb5kdc[18863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346097, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:14:57.590000 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 18982
I20260810 07:14:57.590106 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance
I20260810 07:14:57.590339 16229 external_mini_cluster.cc:1428] Setting key 752be606e5ca8bd6a2bb8810a7870ad6
I20260810 07:14:57.590404 18954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:57.581575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:59741 (local address 127.15.217.126:35641)
0810 07:14:57.581639 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:57.581643 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:57.581886 (+ 243us) server_negotiation.cc:407] Connection header received
0810 07:14:57.582388 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:57.582390 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:57.582401 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:57.582454 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:57.583394 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:57.583901 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:57.584918 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:57.585160 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:57.585219 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:57.587402 (+ 2183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:57.587403 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:57.587404 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:57.587410 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:57.588893 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:57.589274 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:57.589275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:57.589275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:57.589326 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:57.589552 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:57.589553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:57.589553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:57.589664 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:57.589718 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:57.590244 (+ 526us) server_negotiation.cc:299] Negotiation successful
0810 07:14:57.590320 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 07:14:57.590688 19128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:57.581556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35641 (local address 127.15.217.65:59741)
0810 07:14:57.581764 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:57.581803 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:57.582304 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:57.582509 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:57.582513 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:57.583101 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:14:57.583316 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:57.583322 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:57.583981 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:57.583982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:57.584834 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:57.584836 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:57.585237 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:57.585238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:57.585340 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:57.585898 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:14:57.585907 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:14:57.587312 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:14:57.588979 (+ 1667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:57.588987 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:57.588988 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:57.589210 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:57.589387 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:57.589388 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:57.589388 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:57.589494 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:57.589838 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:14:57.589845 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:14:57.590031 (+ 186us) client_negotiation.cc:769] Sending connection context
0810 07:14:57.590229 (+ 198us) client_negotiation.cc:240] Negotiation successful
0810 07:14:57.590420 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":125,"threads_started":1}
I20260810 07:14:57.591238 19125 heartbeater.cc:347] Connected to a master server at 127.15.217.126:35641
I20260810 07:14:57.591356 19125 heartbeater.cc:464] Registering TS with master...
I20260810 07:14:57.591562 19125 heartbeater.cc:511] Master 127.15.217.126:35641 requested a full tablet report, sending...
I20260810 07:14:57.592132 18911 ts_manager.cc:194] Registered new tserver with Master: c32244d835a54308bbd761426ae88c3c (127.15.217.65:44479)
I20260810 07:14:57.592839 18911 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.65@KRBTEST.COM'} at 127.15.217.65:59741
WARNING: no policy specified for kudu/127.15.217.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab.
Entry for principal kudu/127.15.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab.
I20260810 07:14:57.617472 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.66:0
--local_ip_for_outbound_sockets=127.15.217.66
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:35641
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:38807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:14:57.704219 19131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:14:57.704437 19131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:14:57.704481 19131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:14:57.705652 19131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:14:57.705691 19131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:14:57.705724 19131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.66
I20260810 07:14:57.707537 19131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:38807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:35641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.66
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:14:57.707747 19131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:14:57.707953 19131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:14:57.713033 19137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:57.713048 19139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:14:57.713163 19131 server_base.cc:1034] running on GCE node
W20260810 07:14:57.713042 19136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:14:57.713439 19131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:14:57.713693 19131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:14:57.714867 19131 hybrid_clock.cc:648] HybridClock initialized: now 1786346097714851 us; error 42 us; skew 500 ppm
Aug 10 07:14:57 dist-test-slave-10pc krb5kdc[18863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346097, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:14:57.722061 19131 init.cc:377] Logged in from keytab as kudu/127.15.217.66@KRBTEST.COM (short username kudu)
I20260810 07:14:57.722591 19131 webserver.cc:460] Webserver started at http://127.15.217.66:38117/ using document root <none> and password file <none>
I20260810 07:14:57.722867 19131 fs_manager.cc:359] Metadata directory not provided
I20260810 07:14:57.722937 19131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:14:57.723085 19131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:14:57.724188 19131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-1/data/instance:
uuid: "1a435beb083a40bb80692905da700866"
format_stamp: "Formatted at 2026-08-10 07:14:57 on dist-test-slave-10pc"
server_key: "b9d5b2e0bb6252eac1230dbc4357f7aa"
server_key_iv: "74c364efdb517cd7eaccece334d1fd08"
server_key_version: "encryptionkey@0"
I20260810 07:14:57.724623 19131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance:
uuid: "1a435beb083a40bb80692905da700866"
format_stamp: "Formatted at 2026-08-10 07:14:57 on dist-test-slave-10pc"
server_key: "b9d5b2e0bb6252eac1230dbc4357f7aa"
server_key_iv: "74c364efdb517cd7eaccece334d1fd08"
server_key_version: "encryptionkey@0"
I20260810 07:14:57.726061 19131 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:14:57.726858 19149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:57.727038 19131 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 07:14:57.727142 19131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-1/wal
uuid: "1a435beb083a40bb80692905da700866"
format_stamp: "Formatted at 2026-08-10 07:14:57 on dist-test-slave-10pc"
server_key: "b9d5b2e0bb6252eac1230dbc4357f7aa"
server_key_iv: "74c364efdb517cd7eaccece334d1fd08"
server_key_version: "encryptionkey@0"
I20260810 07:14:57.727234 19131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:14:57.737593 19131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:14:57.739639 19131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:14:57.739809 19131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:14:57.740065 19131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:14:57.740411 19131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:14:57.740505 19131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:57.740559 19131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:14:57.740590 19131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:57.746824 19131 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.66:38055
I20260810 07:14:57.746877 19268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.66:38055 every 8 connection(s)
I20260810 07:14:57.747224 19131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
Aug 10 07:14:57 dist-test-slave-10pc krb5kdc[18863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346097, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:14:57.753597 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 19131
I20260810 07:14:57.753719 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance
I20260810 07:14:57.753938 16229 external_mini_cluster.cc:1428] Setting key 93ff98ca914878c0eb092796697ddd80
I20260810 07:14:57.759114 18954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:57.747984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:34425 (local address 127.15.217.126:35641)
0810 07:14:57.748057 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:57.748059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:57.748257 (+ 198us) server_negotiation.cc:407] Connection header received
0810 07:14:57.748858 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:57.748860 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:57.748871 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:57.748926 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:57.749956 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:57.750464 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:57.751454 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:57.751646 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:57.751705 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:57.754127 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:57.754129 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:57.754129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:57.754137 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:57.757551 (+ 3414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:57.758096 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:57.758097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:57.758098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:57.758156 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:57.758460 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:57.758460 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:57.758460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:57.758569 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:57.758618 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:57.758974 (+ 356us) server_negotiation.cc:299] Negotiation successful
0810 07:14:57.759045 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 07:14:57.759479 19271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:57.747979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35641 (local address 127.15.217.66:34425)
0810 07:14:57.748141 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:57.748181 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:57.748759 (+ 578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:57.749012 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:57.749016 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:57.749624 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:14:57.749875 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:57.749881 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:57.750544 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:57.750545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:57.751378 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:57.751380 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:57.751724 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:57.751725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:57.751833 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:57.752329 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:14:57.752337 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:14:57.754038 (+ 1701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:14:57.757644 (+ 3606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:57.757655 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:57.757657 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:57.758017 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:57.758223 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:57.758224 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:57.758225 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:57.758389 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:57.758629 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:14:57.758636 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:14:57.758824 (+ 188us) client_negotiation.cc:769] Sending connection context
0810 07:14:57.758943 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 07:14:57.759143 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
I20260810 07:14:57.759999 19269 heartbeater.cc:347] Connected to a master server at 127.15.217.126:35641
I20260810 07:14:57.760121 19269 heartbeater.cc:464] Registering TS with master...
I20260810 07:14:57.760324 19269 heartbeater.cc:511] Master 127.15.217.126:35641 requested a full tablet report, sending...
I20260810 07:14:57.760849 18911 ts_manager.cc:194] Registered new tserver with Master: 1a435beb083a40bb80692905da700866 (127.15.217.66:38055)
I20260810 07:14:57.761433 18911 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.66@KRBTEST.COM'} at 127.15.217.66:34425
WARNING: no policy specified for kudu/127.15.217.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab.
Entry for principal kudu/127.15.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab.
I20260810 07:14:57.782954 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.67:0
--local_ip_for_outbound_sockets=127.15.217.67
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:35641
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:38807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:14:57.868693 19274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:14:57.868907 19274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:14:57.868932 19274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:14:57.870065 19274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:14:57.870102 19274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:14:57.870128 19274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.67
I20260810 07:14:57.871526 19274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:38807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:35641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.67
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:14:57.871750 19274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:14:57.871939 19274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:14:57.877063 19279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:57.877179 19282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:57.877100 19280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:14:57.877185 19274 server_base.cc:1034] running on GCE node
I20260810 07:14:57.877503 19274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:14:57.877774 19274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:14:57.878955 19274 hybrid_clock.cc:648] HybridClock initialized: now 1786346097878942 us; error 41 us; skew 500 ppm
Aug 10 07:14:57 dist-test-slave-10pc krb5kdc[18863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346097, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:14:57.881507 19274 init.cc:377] Logged in from keytab as kudu/127.15.217.67@KRBTEST.COM (short username kudu)
I20260810 07:14:57.882001 19274 webserver.cc:460] Webserver started at http://127.15.217.67:41029/ using document root <none> and password file <none>
I20260810 07:14:57.882249 19274 fs_manager.cc:359] Metadata directory not provided
I20260810 07:14:57.882318 19274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:14:57.882460 19274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:14:57.883541 19274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-2/data/instance:
uuid: "fe3c7027f6574e1a90ba19d0a4026fa0"
format_stamp: "Formatted at 2026-08-10 07:14:57 on dist-test-slave-10pc"
server_key: "f4be319f9cd21477f679b3e788e4c749"
server_key_iv: "8420753cadbae3c706f1f04e83715f18"
server_key_version: "encryptionkey@0"
I20260810 07:14:57.883934 19274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance:
uuid: "fe3c7027f6574e1a90ba19d0a4026fa0"
format_stamp: "Formatted at 2026-08-10 07:14:57 on dist-test-slave-10pc"
server_key: "f4be319f9cd21477f679b3e788e4c749"
server_key_iv: "8420753cadbae3c706f1f04e83715f18"
server_key_version: "encryptionkey@0"
I20260810 07:14:57.885493 19274 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:14:57.886276 19289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:57.886451 19274 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 07:14:57.886559 19274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-2/wal
uuid: "fe3c7027f6574e1a90ba19d0a4026fa0"
format_stamp: "Formatted at 2026-08-10 07:14:57 on dist-test-slave-10pc"
server_key: "f4be319f9cd21477f679b3e788e4c749"
server_key_iv: "8420753cadbae3c706f1f04e83715f18"
server_key_version: "encryptionkey@0"
I20260810 07:14:57.886632 19274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:14:57.906548 19274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:14:57.909044 19274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:14:57.909225 19274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:14:57.909472 19274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:14:57.909822 19274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:14:57.909878 19274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:57.909914 19274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:14:57.909941 19274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:57.915956 19274 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.67:32777
I20260810 07:14:57.916004 19411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.67:32777 every 8 connection(s)
I20260810 07:14:57.916359 19274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
I20260810 07:14:57.918952 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 19274
I20260810 07:14:57.919056 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance
I20260810 07:14:57.919242 16229 external_mini_cluster.cc:1428] Setting key de941bb5b6f83e5ddc5399cda2ceed63
Aug 10 07:14:57 dist-test-slave-10pc krb5kdc[18863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346097, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:14:57.925843 18954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:57.917165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:60145 (local address 127.15.217.126:35641)
0810 07:14:57.917213 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:57.917215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:57.917446 (+ 231us) server_negotiation.cc:407] Connection header received
0810 07:14:57.917964 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:57.917965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:57.917974 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:57.918035 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:57.919046 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:57.919709 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:57.920817 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:57.921032 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:57.921109 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:57.923300 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:57.923301 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:57.923301 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:57.923308 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:57.924599 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:57.925016 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:57.925017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:57.925017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:57.925071 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:57.925292 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:57.925292 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:57.925293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:57.925395 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:57.925465 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:57.925711 (+ 246us) server_negotiation.cc:299] Negotiation successful
0810 07:14:57.925776 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 07:14:57.926044 19414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:57.917129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35641 (local address 127.15.217.67:60145)
0810 07:14:57.917307 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:57.917354 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:57.917877 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:57.918093 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:57.918097 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:57.918725 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:14:57.918939 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:57.918944 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:57.919820 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:57.919821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:57.920724 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:57.920728 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:57.921122 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:57.921123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:57.921252 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:57.921780 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:14:57.921790 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:14:57.923214 (+ 1424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:14:57.924681 (+ 1467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:57.924688 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:57.924691 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:57.924951 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:57.925127 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:57.925128 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:57.925128 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:57.925239 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:57.925467 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:14:57.925472 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:14:57.925600 (+ 128us) client_negotiation.cc:769] Sending connection context
0810 07:14:57.925694 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 07:14:57.925820 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":99,"threads_started":1}
I20260810 07:14:57.926559 19412 heartbeater.cc:347] Connected to a master server at 127.15.217.126:35641
I20260810 07:14:57.926681 19412 heartbeater.cc:464] Registering TS with master...
I20260810 07:14:57.926908 19412 heartbeater.cc:511] Master 127.15.217.126:35641 requested a full tablet report, sending...
I20260810 07:14:57.927345 18911 ts_manager.cc:194] Registered new tserver with Master: fe3c7027f6574e1a90ba19d0a4026fa0 (127.15.217.67:32777)
I20260810 07:14:57.927733 18911 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.67@KRBTEST.COM'} at 127.15.217.67:60145
I20260810 07:14:57.930840 16229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 07:14:57.944779 18954 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:14:57.943762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47950 (local address 127.15.217.126:35641)
0810 07:14:57.943829 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:57.943831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:57.943940 (+ 109us) server_negotiation.cc:407] Connection header received
0810 07:14:57.943996 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:57.943997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:57.944008 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:57.944075 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:57.944712 (+ 637us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47950: BlockingRecv error: recv got EOF from 127.0.0.1:47950 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 07:14:57.945784 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 18982
I20260810 07:14:57.954026 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 19131
I20260810 07:14:57.964316 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 19274
I20260810 07:14:57.970639 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 18877
2026-08-10T07:14:57Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3224 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:14:58 dist-test-slave-10pc krb5kdc[19425](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:14:58 dist-test-slave-10pc krb5kdc[19425](info): set up 2 sockets
Aug 10 07:14:58 dist-test-slave-10pc krb5kdc[19425](info): commencing operation
krb5kdc: starting...
W20260810 07:14:59.066208 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:14:59 dist-test-slave-10pc krb5kdc[19425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346099, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:14:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:14:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
I20260810 07:14:59.201337 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:42799
--webserver_interface=127.15.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:45055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:42799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:14:59.293715 19440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:14:59.293967 19440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:14:59.293994 19440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:14:59.295331 19440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:14:59.295385 19440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:14:59.295401 19440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:14:59.295467 19440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:14:59.295493 19440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:14:59.297361 19440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:45055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:42799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:42799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:14:59.297652 19440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:14:59.297930 19440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:14:59.303040 19446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:59.302989 19450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:59.302970 19448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:14:59.303285 19440 server_base.cc:1034] running on GCE node
I20260810 07:14:59.303501 19440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:14:59.303761 19440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:14:59.304905 19440 hybrid_clock.cc:648] HybridClock initialized: now 1786346099304890 us; error 29 us; skew 500 ppm
Aug 10 07:14:59 dist-test-slave-10pc krb5kdc[19425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346099, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:14:59.307547 19440 init.cc:377] Logged in from keytab as kudu/127.15.217.126@KRBTEST.COM (short username kudu)
I20260810 07:14:59.307991 19440 webserver.cc:460] Webserver started at http://127.15.217.126:40055/ using document root <none> and password file <none>
I20260810 07:14:59.308234 19440 fs_manager.cc:359] Metadata directory not provided
I20260810 07:14:59.308305 19440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:14:59.308508 19440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:14:59.309643 19440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "9e3a5c1a32da498fa3b89b36f64bedae"
format_stamp: "Formatted at 2026-08-10 07:14:59 on dist-test-slave-10pc"
server_key: "e704b9ea8965be594decf5b807699029"
server_key_iv: "70af5fa27c20e84f0765a23ac5674f96"
server_key_version: "encryptionkey@0"
I20260810 07:14:59.310042 19440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "9e3a5c1a32da498fa3b89b36f64bedae"
format_stamp: "Formatted at 2026-08-10 07:14:59 on dist-test-slave-10pc"
server_key: "e704b9ea8965be594decf5b807699029"
server_key_iv: "70af5fa27c20e84f0765a23ac5674f96"
server_key_version: "encryptionkey@0"
I20260810 07:14:59.311492 19440 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:14:59.312300 19457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:59.312547 19440 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:14:59.312655 19440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "9e3a5c1a32da498fa3b89b36f64bedae"
format_stamp: "Formatted at 2026-08-10 07:14:59 on dist-test-slave-10pc"
server_key: "e704b9ea8965be594decf5b807699029"
server_key_iv: "70af5fa27c20e84f0765a23ac5674f96"
server_key_version: "encryptionkey@0"
I20260810 07:14:59.312742 19440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:14:59.326663 19440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:14:59.329077 19440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:14:59.329279 19440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:14:59.333693 19440 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:42799
I20260810 07:14:59.333729 19512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:42799 every 8 connection(s)
I20260810 07:14:59.334357 19440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:14:59.335052 19513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:14:59.337646 19513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e3a5c1a32da498fa3b89b36f64bedae: Bootstrap starting.
I20260810 07:14:59.337903 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 19440
I20260810 07:14:59.337991 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:14:59.338183 16229 external_mini_cluster.cc:1428] Setting key cd2e93c0a34f947367c6df922d43ba03
I20260810 07:14:59.338428 19513 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e3a5c1a32da498fa3b89b36f64bedae: Neither blocks nor log segments found. Creating new log.
I20260810 07:14:59.338838 19513 log.cc:824] T 00000000000000000000000000000000 P 9e3a5c1a32da498fa3b89b36f64bedae: Log is configured to *not* fsync() on all Append() calls
I20260810 07:14:59.339726 19513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e3a5c1a32da498fa3b89b36f64bedae: No bootstrap required, opened a new log
I20260810 07:14:59.342240 19513 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e3a5c1a32da498fa3b89b36f64bedae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e3a5c1a32da498fa3b89b36f64bedae" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42799 } }
I20260810 07:14:59.342428 19513 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e3a5c1a32da498fa3b89b36f64bedae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:14:59.342480 19513 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e3a5c1a32da498fa3b89b36f64bedae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e3a5c1a32da498fa3b89b36f64bedae, State: Initialized, Role: FOLLOWER
I20260810 07:14:59.342628 19513 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e3a5c1a32da498fa3b89b36f64bedae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e3a5c1a32da498fa3b89b36f64bedae" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42799 } }
I20260810 07:14:59.342730 19513 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e3a5c1a32da498fa3b89b36f64bedae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:14:59.342772 19513 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e3a5c1a32da498fa3b89b36f64bedae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:14:59.342824 19513 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e3a5c1a32da498fa3b89b36f64bedae [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:14:59.343582 19513 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e3a5c1a32da498fa3b89b36f64bedae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e3a5c1a32da498fa3b89b36f64bedae" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42799 } }
I20260810 07:14:59.343700 19513 leader_election.cc:302] T 00000000000000000000000000000000 P 9e3a5c1a32da498fa3b89b36f64bedae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e3a5c1a32da498fa3b89b36f64bedae; no voters:
I20260810 07:14:59.343878 19513 leader_election.cc:288] T 00000000000000000000000000000000 P 9e3a5c1a32da498fa3b89b36f64bedae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:14:59.343950 19518 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e3a5c1a32da498fa3b89b36f64bedae [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:14:59.344146 19518 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e3a5c1a32da498fa3b89b36f64bedae [term 1 LEADER]: Becoming Leader. State: Replica: 9e3a5c1a32da498fa3b89b36f64bedae, State: Running, Role: LEADER
I20260810 07:14:59.344287 19513 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e3a5c1a32da498fa3b89b36f64bedae [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:14:59.344326 19518 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e3a5c1a32da498fa3b89b36f64bedae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e3a5c1a32da498fa3b89b36f64bedae" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42799 } }
I20260810 07:14:59.344754 19518 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e3a5c1a32da498fa3b89b36f64bedae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e3a5c1a32da498fa3b89b36f64bedae. Latest consensus state: current_term: 1 leader_uuid: "9e3a5c1a32da498fa3b89b36f64bedae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e3a5c1a32da498fa3b89b36f64bedae" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42799 } } }
I20260810 07:14:59.344765 19519 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e3a5c1a32da498fa3b89b36f64bedae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e3a5c1a32da498fa3b89b36f64bedae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e3a5c1a32da498fa3b89b36f64bedae" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42799 } } }
I20260810 07:14:59.344961 19519 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e3a5c1a32da498fa3b89b36f64bedae [sys.catalog]: This master's current role is: LEADER
I20260810 07:14:59.345233 19518 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e3a5c1a32da498fa3b89b36f64bedae [sys.catalog]: This master's current role is: LEADER
I20260810 07:14:59.345440 19527 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:14:59.346055 19527 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 07:14:59 dist-test-slave-10pc krb5kdc[19425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346099, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:14:59.348234 19527 catalog_manager.cc:1347] Generated new cluster ID: 8afac90cf54b40f9957f61b92461967e
I20260810 07:14:59.348291 19527 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:14:59.363389 19516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:59.338765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56072 (local address 127.15.217.126:42799)
0810 07:14:59.338990 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:59.339000 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:59.339047 (+ 47us) server_negotiation.cc:407] Connection header received
0810 07:14:59.339658 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:59.339668 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:59.339744 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:59.339904 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:59.340584 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:59.342094 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:59.343150 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:59.343823 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:59.343939 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:59.347233 (+ 3294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:59.347238 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:59.347238 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:59.347252 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:59.361515 (+ 14263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:59.362040 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:59.362043 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:59.362043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:59.362167 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:59.362396 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:59.362397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:59.362397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:59.362713 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:59.362861 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:59.363004 (+ 143us) server_negotiation.cc:299] Negotiation successful
0810 07:14:59.363152 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":102,"threads_started":1}
I20260810 07:14:59.364383 16229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:14:59.381695 19527 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:14:59.382275 19527 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:14:59.383667 19516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:59.378842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56080 (local address 127.15.217.126:42799)
0810 07:14:59.378897 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:59.378900 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:59.378998 (+ 98us) server_negotiation.cc:407] Connection header received
0810 07:14:59.379094 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:59.379095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:59.379107 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:59.379165 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:59.379670 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:59.380678 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:59.381863 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:59.382198 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:59.382240 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:59.383121 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:59.383122 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:59.383123 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:14:59.383345 (+ 222us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:59.383364 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:14:59.383371 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:59.383432 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:59.383524 (+ 92us) server_negotiation.cc:299] Negotiation successful
0810 07:14:59.383567 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":215,"server-negotiator.queue_time_us":25}
I20260810 07:14:59.385581 19527 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e3a5c1a32da498fa3b89b36f64bedae: Generated new TSK 0
I20260810 07:14:59.385783 19527 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:14:59.390856 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 19440
2026-08-10T07:14:59Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1418 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:14:59 dist-test-slave-10pc krb5kdc[19547](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:14:59 dist-test-slave-10pc krb5kdc[19547](info): set up 2 sockets
Aug 10 07:14:59 dist-test-slave-10pc krb5kdc[19547](info): commencing operation
krb5kdc: starting...
W20260810 07:15:00.496062 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:15:00 dist-test-slave-10pc krb5kdc[19547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:15:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:15:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
I20260810 07:15:00.625258 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:46737
--webserver_interface=127.15.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:34311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:46737
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:00.711501 19560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:00.711757 19560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:00.711798 19560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:00.713093 19560 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:00.713171 19560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:00.713196 19560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:00.713225 19560 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:00.713258 19560 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:15:00.714799 19560 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:34311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:46737
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:46737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:00.715097 19560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:00.715350 19560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:00.720323 19566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:00.720350 19565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:00.720345 19568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:00.720738 19560 server_base.cc:1034] running on GCE node
I20260810 07:15:00.721026 19560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:00.721369 19560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:00.722512 19560 hybrid_clock.cc:648] HybridClock initialized: now 1786346100722494 us; error 41 us; skew 500 ppm
Aug 10 07:15:00 dist-test-slave-10pc krb5kdc[19547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346100, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:00.725220 19560 init.cc:377] Logged in from keytab as kudu/127.15.217.126@KRBTEST.COM (short username kudu)
I20260810 07:15:00.725724 19560 webserver.cc:460] Webserver started at http://127.15.217.126:44681/ using document root <none> and password file <none>
I20260810 07:15:00.725981 19560 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:00.726028 19560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:00.726202 19560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:00.727743 19560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "2ea21b53918242b39272ce1749767d54"
format_stamp: "Formatted at 2026-08-10 07:15:00 on dist-test-slave-10pc"
server_key: "fbfb334e5744ccda412c18186622b073"
server_key_iv: "4d41a9ce6bb145271970ccee7fa09b53"
server_key_version: "encryptionkey@0"
I20260810 07:15:00.728132 19560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "2ea21b53918242b39272ce1749767d54"
format_stamp: "Formatted at 2026-08-10 07:15:00 on dist-test-slave-10pc"
server_key: "fbfb334e5744ccda412c18186622b073"
server_key_iv: "4d41a9ce6bb145271970ccee7fa09b53"
server_key_version: "encryptionkey@0"
I20260810 07:15:00.729632 19560 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260810 07:15:00.730443 19576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:00.730638 19560 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 07:15:00.730760 19560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "2ea21b53918242b39272ce1749767d54"
format_stamp: "Formatted at 2026-08-10 07:15:00 on dist-test-slave-10pc"
server_key: "fbfb334e5744ccda412c18186622b073"
server_key_iv: "4d41a9ce6bb145271970ccee7fa09b53"
server_key_version: "encryptionkey@0"
I20260810 07:15:00.730835 19560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:00.748199 19560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:00.750758 19560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:00.750886 19560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:00.755574 19560 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:46737
I20260810 07:15:00.755621 19632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:46737 every 8 connection(s)
I20260810 07:15:00.756059 19560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:15:00.757311 19633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:00.759825 19633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ea21b53918242b39272ce1749767d54: Bootstrap starting.
I20260810 07:15:00.760406 19633 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ea21b53918242b39272ce1749767d54: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:00.760757 19633 log.cc:824] T 00000000000000000000000000000000 P 2ea21b53918242b39272ce1749767d54: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:00.761304 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 19560
I20260810 07:15:00.761428 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:15:00.761586 19633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ea21b53918242b39272ce1749767d54: No bootstrap required, opened a new log
I20260810 07:15:00.761608 16229 external_mini_cluster.cc:1428] Setting key d1d119647d6ee6f06b0632324c089a59
I20260810 07:15:00.763974 19633 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ea21b53918242b39272ce1749767d54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea21b53918242b39272ce1749767d54" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 46737 } }
I20260810 07:15:00.764256 19633 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ea21b53918242b39272ce1749767d54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:00.764328 19633 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ea21b53918242b39272ce1749767d54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ea21b53918242b39272ce1749767d54, State: Initialized, Role: FOLLOWER
I20260810 07:15:00.764513 19633 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ea21b53918242b39272ce1749767d54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea21b53918242b39272ce1749767d54" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 46737 } }
I20260810 07:15:00.764602 19633 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ea21b53918242b39272ce1749767d54 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:15:00.764643 19633 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ea21b53918242b39272ce1749767d54 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:15:00.764690 19633 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ea21b53918242b39272ce1749767d54 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:00.765437 19633 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ea21b53918242b39272ce1749767d54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea21b53918242b39272ce1749767d54" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 46737 } }
I20260810 07:15:00.765553 19633 leader_election.cc:302] T 00000000000000000000000000000000 P 2ea21b53918242b39272ce1749767d54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ea21b53918242b39272ce1749767d54; no voters:
I20260810 07:15:00.765709 19633 leader_election.cc:288] T 00000000000000000000000000000000 P 2ea21b53918242b39272ce1749767d54 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:15:00.765772 19638 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ea21b53918242b39272ce1749767d54 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:00.765974 19638 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ea21b53918242b39272ce1749767d54 [term 1 LEADER]: Becoming Leader. State: Replica: 2ea21b53918242b39272ce1749767d54, State: Running, Role: LEADER
I20260810 07:15:00.765980 19633 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ea21b53918242b39272ce1749767d54 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:00.766091 19638 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ea21b53918242b39272ce1749767d54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea21b53918242b39272ce1749767d54" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 46737 } }
I20260810 07:15:00.766845 19640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ea21b53918242b39272ce1749767d54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ea21b53918242b39272ce1749767d54. Latest consensus state: current_term: 1 leader_uuid: "2ea21b53918242b39272ce1749767d54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea21b53918242b39272ce1749767d54" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 46737 } } }
I20260810 07:15:00.766943 19640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ea21b53918242b39272ce1749767d54 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:00.767186 19639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ea21b53918242b39272ce1749767d54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ea21b53918242b39272ce1749767d54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ea21b53918242b39272ce1749767d54" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 46737 } } }
I20260810 07:15:00.767325 19639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ea21b53918242b39272ce1749767d54 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:00.767486 19648 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:00.768137 19648 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:15:00.769904 19648 catalog_manager.cc:1347] Generated new cluster ID: 5711d0bf66024941bb3b4288b7998c50
I20260810 07:15:00.769965 19648 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 07:15:00 dist-test-slave-10pc krb5kdc[19547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:15:00.774331 19637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:00.762189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50766 (local address 127.15.217.126:46737)
0810 07:15:00.762430 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:00.762441 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:00.762476 (+ 35us) server_negotiation.cc:407] Connection header received
0810 07:15:00.763130 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:00.763152 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:00.763222 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:00.763381 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:00.763976 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:00.765772 (+ 1796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:00.766883 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:00.767531 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:00.767640 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:00.770768 (+ 3128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:00.770772 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:00.770773 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:00.770786 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:00.772679 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:00.773056 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:00.773059 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:00.773059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:00.773159 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:00.773380 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:00.773382 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:00.773382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:00.773633 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:00.773814 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:00.773902 (+ 88us) server_negotiation.cc:299] Negotiation successful
0810 07:15:00.774045 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":115,"threads_started":1}
I20260810 07:15:00.779420 19648 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:15:00.779989 19648 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:00.785866 19648 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ea21b53918242b39272ce1749767d54: Generated new TSK 0
I20260810 07:15:00.786056 19648 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab.
Entry for principal kudu/127.15.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab.
I20260810 07:15:00.817741 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.65:0
--local_ip_for_outbound_sockets=127.15.217.65
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:46737
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:34311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:00.904397 19663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:00.904636 19663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:00.904668 19663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:00.905781 19663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:00.905817 19663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:00.905843 19663 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.65
I20260810 07:15:00.907233 19663 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:34311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:46737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.65
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:00.907426 19663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:00.907615 19663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:00.913058 19671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:00.913057 19674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:00.913090 19672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:00.913094 19663 server_base.cc:1034] running on GCE node
I20260810 07:15:00.913518 19663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:00.913794 19663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:00.914970 19663 hybrid_clock.cc:648] HybridClock initialized: now 1786346100914944 us; error 59 us; skew 500 ppm
Aug 10 07:15:00 dist-test-slave-10pc krb5kdc[19547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346100, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:00.917652 19663 init.cc:377] Logged in from keytab as kudu/127.15.217.65@KRBTEST.COM (short username kudu)
I20260810 07:15:00.918148 19663 webserver.cc:460] Webserver started at http://127.15.217.65:41871/ using document root <none> and password file <none>
I20260810 07:15:00.918390 19663 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:00.918462 19663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:00.918622 19663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:00.919770 19663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-0/data/instance:
uuid: "b9a4252caccc48f3b9c4b07473e26d5d"
format_stamp: "Formatted at 2026-08-10 07:15:00 on dist-test-slave-10pc"
server_key: "6239ff6867a33dfd2992b0a5539eadd8"
server_key_iv: "559b8519e21ed3add5ffcaf2725769e4"
server_key_version: "encryptionkey@0"
I20260810 07:15:00.920186 19663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance:
uuid: "b9a4252caccc48f3b9c4b07473e26d5d"
format_stamp: "Formatted at 2026-08-10 07:15:00 on dist-test-slave-10pc"
server_key: "6239ff6867a33dfd2992b0a5539eadd8"
server_key_iv: "559b8519e21ed3add5ffcaf2725769e4"
server_key_version: "encryptionkey@0"
I20260810 07:15:00.921746 19663 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 07:15:00.922580 19683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:00.922814 19663 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:00.922931 19663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-0/wal
uuid: "b9a4252caccc48f3b9c4b07473e26d5d"
format_stamp: "Formatted at 2026-08-10 07:15:00 on dist-test-slave-10pc"
server_key: "6239ff6867a33dfd2992b0a5539eadd8"
server_key_iv: "559b8519e21ed3add5ffcaf2725769e4"
server_key_version: "encryptionkey@0"
I20260810 07:15:00.923012 19663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:00.945933 19663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:00.948012 19663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:00.948215 19663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:00.948524 19663 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:00.948900 19663 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:00.948961 19663 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:00.948997 19663 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:00.949026 19663 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:00.955215 19663 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.65:45281
I20260810 07:15:00.955305 19808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.65:45281 every 8 connection(s)
I20260810 07:15:00.955675 19663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
Aug 10 07:15:00 dist-test-slave-10pc krb5kdc[19547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346100, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:15:00.964193 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 19663
I20260810 07:15:00.964315 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance
I20260810 07:15:00.964648 16229 external_mini_cluster.cc:1428] Setting key 4813d5424d8917d703b89a8f79b487f2
I20260810 07:15:00.965821 19637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:00.956526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:33519 (local address 127.15.217.126:46737)
0810 07:15:00.956583 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:00.956585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:00.956884 (+ 299us) server_negotiation.cc:407] Connection header received
0810 07:15:00.957404 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:00.957406 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:00.957417 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:00.957469 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:00.958457 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:00.958959 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:00.959956 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:00.960160 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:00.960210 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:00.962562 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:00.962563 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:00.962564 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:00.962570 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:00.963873 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:00.964385 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:00.964386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:00.964386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:00.964480 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:00.964837 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:00.964838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:00.964838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:00.965011 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:00.965089 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:00.965566 (+ 477us) server_negotiation.cc:299] Negotiation successful
0810 07:15:00.965677 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 07:15:00.965828 19811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:00.956530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:46737 (local address 127.15.217.65:33519)
0810 07:15:00.956746 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:00.956785 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:00.957317 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:00.957533 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:00.957536 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:00.958160 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:00.958372 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:00.958378 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:00.959042 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:00.959043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:00.959879 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:00.959881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:00.960235 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:00.960235 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:00.960338 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:00.961012 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:00.961022 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:00.962467 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:00.963962 (+ 1495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:00.963971 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:00.963973 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:00.964312 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:00.964571 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:00.964573 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:00.964573 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:00.964752 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:00.965293 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:00.965300 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:00.965428 (+ 128us) client_negotiation.cc:769] Sending connection context
0810 07:15:00.965519 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 07:15:00.965649 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":104,"threads_started":1}
I20260810 07:15:00.966380 19809 heartbeater.cc:347] Connected to a master server at 127.15.217.126:46737
I20260810 07:15:00.966493 19809 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:00.966717 19809 heartbeater.cc:511] Master 127.15.217.126:46737 requested a full tablet report, sending...
I20260810 07:15:00.967347 19594 ts_manager.cc:194] Registered new tserver with Master: b9a4252caccc48f3b9c4b07473e26d5d (127.15.217.65:45281)
I20260810 07:15:00.968308 19594 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.65@KRBTEST.COM'} at 127.15.217.65:33519
WARNING: no policy specified for kudu/127.15.217.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab.
Entry for principal kudu/127.15.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab.
I20260810 07:15:00.991745 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.66:0
--local_ip_for_outbound_sockets=127.15.217.66
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:46737
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:34311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:01.078457 19814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:01.078685 19814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:01.078711 19814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:01.079815 19814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:01.079850 19814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:01.079877 19814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.66
I20260810 07:15:01.081305 19814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:34311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:46737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.66
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:01.081513 19814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:01.081715 19814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:01.086974 19820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:01.086998 19822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:01.087097 19819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:01.087347 19814 server_base.cc:1034] running on GCE node
I20260810 07:15:01.087589 19814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:01.087855 19814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:01.089003 19814 hybrid_clock.cc:648] HybridClock initialized: now 1786346101088987 us; error 28 us; skew 500 ppm
Aug 10 07:15:01 dist-test-slave-10pc krb5kdc[19547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346101, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:01.091583 19814 init.cc:377] Logged in from keytab as kudu/127.15.217.66@KRBTEST.COM (short username kudu)
I20260810 07:15:01.092048 19814 webserver.cc:460] Webserver started at http://127.15.217.66:40281/ using document root <none> and password file <none>
I20260810 07:15:01.092319 19814 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:01.092367 19814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:01.092581 19814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:01.093721 19814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-1/data/instance:
uuid: "aab3eb76f2fb44fe868a98d56f95d11b"
format_stamp: "Formatted at 2026-08-10 07:15:01 on dist-test-slave-10pc"
server_key: "59a66ccac2cd3e1abf548caa0b7ba7ac"
server_key_iv: "c82e7ceea67bf11052b72cf8d6a7971f"
server_key_version: "encryptionkey@0"
I20260810 07:15:01.094107 19814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance:
uuid: "aab3eb76f2fb44fe868a98d56f95d11b"
format_stamp: "Formatted at 2026-08-10 07:15:01 on dist-test-slave-10pc"
server_key: "59a66ccac2cd3e1abf548caa0b7ba7ac"
server_key_iv: "c82e7ceea67bf11052b72cf8d6a7971f"
server_key_version: "encryptionkey@0"
I20260810 07:15:01.095534 19814 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:15:01.096297 19830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:01.096519 19814 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:01.096625 19814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-1/wal
uuid: "aab3eb76f2fb44fe868a98d56f95d11b"
format_stamp: "Formatted at 2026-08-10 07:15:01 on dist-test-slave-10pc"
server_key: "59a66ccac2cd3e1abf548caa0b7ba7ac"
server_key_iv: "c82e7ceea67bf11052b72cf8d6a7971f"
server_key_version: "encryptionkey@0"
I20260810 07:15:01.096715 19814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:01.107381 19814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:01.109470 19814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:01.109589 19814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:01.109788 19814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:01.110127 19814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:01.110170 19814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:01.110224 19814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:01.110258 19814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:01.116267 19814 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.66:44373
I20260810 07:15:01.116338 19955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.66:44373 every 8 connection(s)
I20260810 07:15:01.116735 19814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
I20260810 07:15:01.117740 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 19814
I20260810 07:15:01.117865 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance
I20260810 07:15:01.118100 16229 external_mini_cluster.cc:1428] Setting key 738c46e0e8e71430957ea68021518d86
Aug 10 07:15:01 dist-test-slave-10pc krb5kdc[19547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346101, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:15:01.127691 19637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:01.117510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:57971 (local address 127.15.217.126:46737)
0810 07:15:01.117595 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:01.117597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:01.117864 (+ 267us) server_negotiation.cc:407] Connection header received
0810 07:15:01.118630 (+ 766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:01.118632 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:01.118643 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:01.118727 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:01.120121 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:01.120713 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:01.121834 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:01.122165 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:01.122245 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:01.124489 (+ 2244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:01.124490 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:01.124491 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:01.124500 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:01.126227 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:01.126626 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:01.126628 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:01.126628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:01.126708 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:01.126950 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:01.126951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:01.126952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:01.127115 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:01.127190 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:01.127454 (+ 264us) server_negotiation.cc:299] Negotiation successful
0810 07:15:01.127559 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 07:15:01.127707 19958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:01.117475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:46737 (local address 127.15.217.66:57971)
0810 07:15:01.117699 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:01.117754 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:01.118517 (+ 763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:01.118822 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:01.118826 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:01.119712 (+ 886us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:01.120013 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:01.120021 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:01.120842 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:01.120844 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:01.121742 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:01.121744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:01.122256 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:01.122257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:01.122365 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:01.122945 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:01.122954 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:01.124364 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:01.126311 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:01.126319 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:01.126321 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:01.126543 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:01.126781 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:01.126781 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:01.126782 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:01.126881 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:01.127176 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:01.127182 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:01.127306 (+ 124us) client_negotiation.cc:769] Sending connection context
0810 07:15:01.127395 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 07:15:01.127523 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":115,"threads_started":1}
I20260810 07:15:01.128285 19956 heartbeater.cc:347] Connected to a master server at 127.15.217.126:46737
I20260810 07:15:01.128403 19956 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:01.128635 19956 heartbeater.cc:511] Master 127.15.217.126:46737 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.15.217.67@KRBTEST.COM; defaulting to no policy
I20260810 07:15:01.129102 19594 ts_manager.cc:194] Registered new tserver with Master: aab3eb76f2fb44fe868a98d56f95d11b (127.15.217.66:44373)
I20260810 07:15:01.129491 19594 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.66@KRBTEST.COM'} at 127.15.217.66:57971
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab.
Entry for principal kudu/127.15.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab.
I20260810 07:15:01.148046 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.67:0
--local_ip_for_outbound_sockets=127.15.217.67
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:46737
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:34311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:01.234973 19961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:01.235195 19961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:01.235220 19961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:01.236339 19961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:01.236373 19961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:01.236399 19961 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.67
I20260810 07:15:01.238034 19961 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:34311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:46737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.67
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:01.238252 19961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:01.238447 19961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:15:01.243747 19961 server_base.cc:1034] running on GCE node
W20260810 07:15:01.243690 19966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:01.243646 19969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:01.243646 19967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:01.244131 19961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:01.244347 19961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:01.245487 19961 hybrid_clock.cc:648] HybridClock initialized: now 1786346101245469 us; error 33 us; skew 500 ppm
Aug 10 07:15:01 dist-test-slave-10pc krb5kdc[19547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346101, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:01.248083 19961 init.cc:377] Logged in from keytab as kudu/127.15.217.67@KRBTEST.COM (short username kudu)
I20260810 07:15:01.248628 19961 webserver.cc:460] Webserver started at http://127.15.217.67:36375/ using document root <none> and password file <none>
I20260810 07:15:01.248888 19961 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:01.248934 19961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:01.249087 19961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:01.250196 19961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-2/data/instance:
uuid: "7d24e112f5fb49c9bdeff5bef7c7dd33"
format_stamp: "Formatted at 2026-08-10 07:15:01 on dist-test-slave-10pc"
server_key: "a3f8455476adc2ff28e845345ee5b3a8"
server_key_iv: "bccbc1f843bccd706a4543f7a37757b3"
server_key_version: "encryptionkey@0"
I20260810 07:15:01.250576 19961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance:
uuid: "7d24e112f5fb49c9bdeff5bef7c7dd33"
format_stamp: "Formatted at 2026-08-10 07:15:01 on dist-test-slave-10pc"
server_key: "a3f8455476adc2ff28e845345ee5b3a8"
server_key_iv: "bccbc1f843bccd706a4543f7a37757b3"
server_key_version: "encryptionkey@0"
I20260810 07:15:01.251999 19961 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:15:01.252877 19979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:01.253086 19961 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 07:15:01.253186 19961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-2/wal
uuid: "7d24e112f5fb49c9bdeff5bef7c7dd33"
format_stamp: "Formatted at 2026-08-10 07:15:01 on dist-test-slave-10pc"
server_key: "a3f8455476adc2ff28e845345ee5b3a8"
server_key_iv: "bccbc1f843bccd706a4543f7a37757b3"
server_key_version: "encryptionkey@0"
I20260810 07:15:01.253266 19961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:01.266314 19961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:01.268790 19961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:01.268953 19961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:01.269198 19961 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:01.269541 19961 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:01.269598 19961 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:01.269640 19961 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:01.269668 19961 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:01.275924 19961 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.67:38139
I20260810 07:15:01.275979 20098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.67:38139 every 8 connection(s)
I20260810 07:15:01.276374 19961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
Aug 10 07:15:01 dist-test-slave-10pc krb5kdc[19547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346101, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:15:01.284099 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 19961
I20260810 07:15:01.284189 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance
I20260810 07:15:01.284427 16229 external_mini_cluster.cc:1428] Setting key 89d26f7e5c87e8d502c26f1e74cf9982
I20260810 07:15:01.286561 19637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:01.277166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:60341 (local address 127.15.217.126:46737)
0810 07:15:01.277213 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:01.277215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:01.277442 (+ 227us) server_negotiation.cc:407] Connection header received
0810 07:15:01.277973 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:01.277974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:01.277984 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:01.278044 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:01.279038 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:01.279544 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:01.280656 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:01.280849 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:01.280905 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:01.283294 (+ 2389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:01.283295 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:01.283295 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:01.283302 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:01.284873 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:01.285415 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:01.285416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:01.285417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:01.285486 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:01.285833 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:01.285834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:01.285835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:01.286008 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:01.286085 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:01.286414 (+ 329us) server_negotiation.cc:299] Negotiation successful
0810 07:15:01.286482 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 07:15:01.286653 20101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:01.277138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:46737 (local address 127.15.217.67:60341)
0810 07:15:01.277318 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:01.277362 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:01.277881 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:01.278132 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:01.278136 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:01.278740 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:01.278953 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:01.278958 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:01.279635 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:01.279636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:01.280576 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:01.280579 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:01.280926 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:01.280927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:01.281061 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:01.281649 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:01.281659 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:01.283182 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:01.284972 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:01.284981 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:01.284983 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:01.285340 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:01.285575 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:01.285576 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:01.285577 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:01.285745 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:01.286084 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:01.286091 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:01.286224 (+ 133us) client_negotiation.cc:769] Sending connection context
0810 07:15:01.286313 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 07:15:01.286445 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":102,"threads_started":1}
I20260810 07:15:01.287153 20099 heartbeater.cc:347] Connected to a master server at 127.15.217.126:46737
I20260810 07:15:01.287258 20099 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:01.287421 20099 heartbeater.cc:511] Master 127.15.217.126:46737 requested a full tablet report, sending...
I20260810 07:15:01.287806 19594 ts_manager.cc:194] Registered new tserver with Master: 7d24e112f5fb49c9bdeff5bef7c7dd33 (127.15.217.67:38139)
I20260810 07:15:01.288189 19594 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.67@KRBTEST.COM'} at 127.15.217.67:60341
I20260810 07:15:01.296824 16229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 07:15:01 dist-test-slave-10pc krb5kdc[19547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346101, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 10 07:15:01 dist-test-slave-10pc krb5kdc[19547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346101, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:15:01.321854 19637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:01.314483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50776 (local address 127.15.217.126:46737)
0810 07:15:01.314555 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:01.314557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:01.314689 (+ 132us) server_negotiation.cc:407] Connection header received
0810 07:15:01.314747 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:01.314748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:01.314759 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:01.314811 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:01.315391 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:01.315892 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:01.316691 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:01.316876 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:01.316929 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:01.319309 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:01.319310 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:01.319311 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:01.319318 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:01.320851 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:01.321167 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:01.321168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:01.321168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:01.321222 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:01.321389 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:01.321390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:01.321390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:01.321546 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:01.321626 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:01.321720 (+ 94us) server_negotiation.cc:299] Negotiation successful
0810 07:15:01.321782 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260810 07:15:01.322149 19594 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:50776
I20260810 07:15:01.322635 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 19663
I20260810 07:15:01.330183 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 19814
I20260810 07:15:01.338296 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 19961
I20260810 07:15:01.345655 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 19560
2026-08-10T07:15:01Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1957 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:15:01 dist-test-slave-10pc krb5kdc[20113](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:15:01 dist-test-slave-10pc krb5kdc[20113](info): set up 2 sockets
Aug 10 07:15:01 dist-test-slave-10pc krb5kdc[20113](info): commencing operation
krb5kdc: starting...
W20260810 07:15:02.443954 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:15:02 dist-test-slave-10pc krb5kdc[20113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:15:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:15:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
I20260810 07:15:02.571842 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:39469
--webserver_interface=127.15.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:33557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:39469
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:02.658281 20127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:02.658506 20127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:02.658531 20127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:02.659758 20127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:02.659812 20127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:02.659826 20127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:02.659840 20127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:02.659853 20127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:15:02.661607 20127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:33557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:39469
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:39469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:02.661903 20127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:02.662194 20127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:02.667383 20135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:02.667419 20132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:02.667431 20133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:02.667572 20127 server_base.cc:1034] running on GCE node
I20260810 07:15:02.667821 20127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:02.668203 20127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:02.669410 20127 hybrid_clock.cc:648] HybridClock initialized: now 1786346102669390 us; error 31 us; skew 500 ppm
Aug 10 07:15:02 dist-test-slave-10pc krb5kdc[20113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346102, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:02.671965 20127 init.cc:377] Logged in from keytab as kudu/127.15.217.126@KRBTEST.COM (short username kudu)
I20260810 07:15:02.672402 20127 webserver.cc:460] Webserver started at http://127.15.217.126:42455/ using document root <none> and password file <none>
I20260810 07:15:02.672657 20127 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:02.672706 20127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:02.672816 20127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:02.673914 20127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "9ffcb36a7d8e48ff8c5cc0250bad9cb5"
format_stamp: "Formatted at 2026-08-10 07:15:02 on dist-test-slave-10pc"
server_key: "4d0050a91f4959edc9cd162655b8a09e"
server_key_iv: "978187f4594db679b24a266091b97342"
server_key_version: "encryptionkey@0"
I20260810 07:15:02.674254 20127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "9ffcb36a7d8e48ff8c5cc0250bad9cb5"
format_stamp: "Formatted at 2026-08-10 07:15:02 on dist-test-slave-10pc"
server_key: "4d0050a91f4959edc9cd162655b8a09e"
server_key_iv: "978187f4594db679b24a266091b97342"
server_key_version: "encryptionkey@0"
I20260810 07:15:02.675607 20127 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:15:02.676348 20142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:02.676540 20127 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:02.676651 20127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "9ffcb36a7d8e48ff8c5cc0250bad9cb5"
format_stamp: "Formatted at 2026-08-10 07:15:02 on dist-test-slave-10pc"
server_key: "4d0050a91f4959edc9cd162655b8a09e"
server_key_iv: "978187f4594db679b24a266091b97342"
server_key_version: "encryptionkey@0"
I20260810 07:15:02.676735 20127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:02.694804 20127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:02.697085 20127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:02.697291 20127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:02.701682 20127 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:39469
I20260810 07:15:02.701743 20202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:39469 every 8 connection(s)
I20260810 07:15:02.702334 20127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:15:02.703136 20203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:02.705735 20203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ffcb36a7d8e48ff8c5cc0250bad9cb5: Bootstrap starting.
I20260810 07:15:02.706408 20203 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ffcb36a7d8e48ff8c5cc0250bad9cb5: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:02.706748 20203 log.cc:824] T 00000000000000000000000000000000 P 9ffcb36a7d8e48ff8c5cc0250bad9cb5: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:02.707537 20203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ffcb36a7d8e48ff8c5cc0250bad9cb5: No bootstrap required, opened a new log
I20260810 07:15:02.708004 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 20127
I20260810 07:15:02.708138 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:15:02.708329 16229 external_mini_cluster.cc:1428] Setting key 672a7a83356373c7e3e73c0c7f928ab4
I20260810 07:15:02.710314 20203 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ffcb36a7d8e48ff8c5cc0250bad9cb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ffcb36a7d8e48ff8c5cc0250bad9cb5" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 39469 } }
I20260810 07:15:02.710520 20203 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ffcb36a7d8e48ff8c5cc0250bad9cb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:02.710568 20203 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ffcb36a7d8e48ff8c5cc0250bad9cb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ffcb36a7d8e48ff8c5cc0250bad9cb5, State: Initialized, Role: FOLLOWER
I20260810 07:15:02.710723 20203 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ffcb36a7d8e48ff8c5cc0250bad9cb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ffcb36a7d8e48ff8c5cc0250bad9cb5" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 39469 } }
I20260810 07:15:02.710824 20203 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ffcb36a7d8e48ff8c5cc0250bad9cb5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:15:02.710861 20203 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ffcb36a7d8e48ff8c5cc0250bad9cb5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:15:02.710917 20203 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ffcb36a7d8e48ff8c5cc0250bad9cb5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:02.711632 20203 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ffcb36a7d8e48ff8c5cc0250bad9cb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ffcb36a7d8e48ff8c5cc0250bad9cb5" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 39469 } }
I20260810 07:15:02.711735 20203 leader_election.cc:302] T 00000000000000000000000000000000 P 9ffcb36a7d8e48ff8c5cc0250bad9cb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ffcb36a7d8e48ff8c5cc0250bad9cb5; no voters:
I20260810 07:15:02.711937 20203 leader_election.cc:288] T 00000000000000000000000000000000 P 9ffcb36a7d8e48ff8c5cc0250bad9cb5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:15:02.711972 20210 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ffcb36a7d8e48ff8c5cc0250bad9cb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:02.712136 20210 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ffcb36a7d8e48ff8c5cc0250bad9cb5 [term 1 LEADER]: Becoming Leader. State: Replica: 9ffcb36a7d8e48ff8c5cc0250bad9cb5, State: Running, Role: LEADER
I20260810 07:15:02.712245 20203 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ffcb36a7d8e48ff8c5cc0250bad9cb5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:02.712337 20210 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ffcb36a7d8e48ff8c5cc0250bad9cb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ffcb36a7d8e48ff8c5cc0250bad9cb5" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 39469 } }
I20260810 07:15:02.712776 20212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ffcb36a7d8e48ff8c5cc0250bad9cb5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ffcb36a7d8e48ff8c5cc0250bad9cb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ffcb36a7d8e48ff8c5cc0250bad9cb5" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 39469 } } }
I20260810 07:15:02.712934 20212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ffcb36a7d8e48ff8c5cc0250bad9cb5 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:02.712766 20210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ffcb36a7d8e48ff8c5cc0250bad9cb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ffcb36a7d8e48ff8c5cc0250bad9cb5. Latest consensus state: current_term: 1 leader_uuid: "9ffcb36a7d8e48ff8c5cc0250bad9cb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ffcb36a7d8e48ff8c5cc0250bad9cb5" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 39469 } } }
I20260810 07:15:02.713271 20210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ffcb36a7d8e48ff8c5cc0250bad9cb5 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:02.713261 20219 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:02.713943 20219 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:15:02.716735 20219 catalog_manager.cc:1347] Generated new cluster ID: d3d33ee9abec4929b37496b2b1a7d777
I20260810 07:15:02.716827 20219 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 07:15:02 dist-test-slave-10pc krb5kdc[20113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:15:02.721670 20209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:02.708935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39344 (local address 127.15.217.126:39469)
0810 07:15:02.709121 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:02.709130 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:02.709157 (+ 27us) server_negotiation.cc:407] Connection header received
0810 07:15:02.710240 (+ 1083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:02.710249 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:02.710306 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:02.710432 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:02.711320 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.712578 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.713706 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.714341 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.714453 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:02.717438 (+ 2985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:02.717444 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:02.717444 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:02.717458 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:02.719678 (+ 2220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:02.720125 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:02.720130 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:02.720130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:02.720318 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:02.720594 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:02.720595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:02.720596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:02.720975 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:02.721166 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:02.721259 (+ 93us) server_negotiation.cc:299] Negotiation successful
0810 07:15:02.721420 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":82,"threads_started":1}
I20260810 07:15:02.724361 20219 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:15:02.724937 20219 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:02.730072 20219 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ffcb36a7d8e48ff8c5cc0250bad9cb5: Generated new TSK 0
I20260810 07:15:02.730284 20219 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab.
Entry for principal kudu/127.15.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab.
I20260810 07:15:02.759590 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.65:0
--local_ip_for_outbound_sockets=127.15.217.65
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:39469
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:33557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:02.848840 20231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:02.849056 20231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:02.849090 20231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:02.850272 20231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:02.850313 20231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:02.850339 20231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.65
I20260810 07:15:02.851866 20231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:33557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:39469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.65
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:02.852152 20231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:02.852412 20231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:02.857752 20236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:02.857790 20237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:02.857796 20239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:02.857836 20231 server_base.cc:1034] running on GCE node
I20260810 07:15:02.858333 20231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:02.858599 20231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:02.859745 20231 hybrid_clock.cc:648] HybridClock initialized: now 1786346102859733 us; error 26 us; skew 500 ppm
Aug 10 07:15:02 dist-test-slave-10pc krb5kdc[20113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346102, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:02.867203 20231 init.cc:377] Logged in from keytab as kudu/127.15.217.65@KRBTEST.COM (short username kudu)
I20260810 07:15:02.867700 20231 webserver.cc:460] Webserver started at http://127.15.217.65:41793/ using document root <none> and password file <none>
I20260810 07:15:02.867935 20231 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:02.867983 20231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:02.868139 20231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:02.869410 20231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-0/data/instance:
uuid: "5400854ec0804f17885e22d678e8d9aa"
format_stamp: "Formatted at 2026-08-10 07:15:02 on dist-test-slave-10pc"
server_key: "80946c3423aa1924ad6ad70d70dbbf10"
server_key_iv: "3c943ac1e6910c632452f69ea9cfc35b"
server_key_version: "encryptionkey@0"
I20260810 07:15:02.869798 20231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance:
uuid: "5400854ec0804f17885e22d678e8d9aa"
format_stamp: "Formatted at 2026-08-10 07:15:02 on dist-test-slave-10pc"
server_key: "80946c3423aa1924ad6ad70d70dbbf10"
server_key_iv: "3c943ac1e6910c632452f69ea9cfc35b"
server_key_version: "encryptionkey@0"
I20260810 07:15:02.871268 20231 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 07:15:02.872089 20246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:02.872298 20231 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:15:02.872399 20231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-0/wal
uuid: "5400854ec0804f17885e22d678e8d9aa"
format_stamp: "Formatted at 2026-08-10 07:15:02 on dist-test-slave-10pc"
server_key: "80946c3423aa1924ad6ad70d70dbbf10"
server_key_iv: "3c943ac1e6910c632452f69ea9cfc35b"
server_key_version: "encryptionkey@0"
I20260810 07:15:02.872522 20231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:02.886439 20231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:02.888695 20231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:02.888849 20231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:02.889112 20231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:02.889457 20231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:02.889493 20231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:02.889516 20231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:02.889559 20231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:02.895812 20231 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.65:39117
I20260810 07:15:02.895864 20369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.65:39117 every 8 connection(s)
I20260810 07:15:02.896221 20231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
Aug 10 07:15:02 dist-test-slave-10pc krb5kdc[20113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346102, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:15:02.906227 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 20231
I20260810 07:15:02.906344 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance
I20260810 07:15:02.906533 16229 external_mini_cluster.cc:1428] Setting key aabe461e0980330e8740fd275af1953a
I20260810 07:15:02.906935 20209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:02.897190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:55667 (local address 127.15.217.126:39469)
0810 07:15:02.897290 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:02.897293 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:02.897531 (+ 238us) server_negotiation.cc:407] Connection header received
0810 07:15:02.898601 (+ 1070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:02.898603 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:02.898616 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:02.898673 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:02.899403 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.899951 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.901043 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.901262 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.901341 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:02.903627 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:02.903629 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:02.903629 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:02.903638 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:02.905275 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:02.905768 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:02.905769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:02.905770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:02.905839 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:02.906149 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:02.906150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:02.906150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:02.906289 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:02.906373 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:02.906725 (+ 352us) server_negotiation.cc:299] Negotiation successful
0810 07:15:02.906829 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 07:15:02.907156 20372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:02.897104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:39469 (local address 127.15.217.65:55667)
0810 07:15:02.897352 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:02.897402 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:02.898471 (+ 1069us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:02.898753 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:02.898758 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:02.899074 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:02.899291 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:02.899297 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.900059 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.900060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:02.900942 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:02.900945 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.901353 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.901354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:02.901497 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:02.902027 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:02.902036 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:02.903513 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:02.905395 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:02.905404 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:02.905407 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:02.905678 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:02.905924 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:02.905925 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:02.905925 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:02.906072 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:02.906365 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:02.906373 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:02.906553 (+ 180us) client_negotiation.cc:769] Sending connection context
0810 07:15:02.906680 (+ 127us) client_negotiation.cc:240] Negotiation successful
0810 07:15:02.906867 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":92,"threads_started":1}
I20260810 07:15:02.907805 20370 heartbeater.cc:347] Connected to a master server at 127.15.217.126:39469
I20260810 07:15:02.907929 20370 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:02.908161 20370 heartbeater.cc:511] Master 127.15.217.126:39469 requested a full tablet report, sending...
I20260810 07:15:02.908782 20160 ts_manager.cc:194] Registered new tserver with Master: 5400854ec0804f17885e22d678e8d9aa (127.15.217.65:39117)
I20260810 07:15:02.909593 20160 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.65@KRBTEST.COM'} at 127.15.217.65:55667
WARNING: no policy specified for kudu/127.15.217.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab.
Entry for principal kudu/127.15.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab.
I20260810 07:15:02.936137 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.66:0
--local_ip_for_outbound_sockets=127.15.217.66
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:39469
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:33557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:03.023103 20375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:03.023350 20375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:03.023388 20375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:03.024567 20375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:03.024636 20375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:03.024677 20375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.66
I20260810 07:15:03.026214 20375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:33557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:39469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.66
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:03.026496 20375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:03.026747 20375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:03.032009 20381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:03.031952 20383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:03.031952 20380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:03.032784 20375 server_base.cc:1034] running on GCE node
I20260810 07:15:03.033105 20375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:03.033326 20375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:03.034492 20375 hybrid_clock.cc:648] HybridClock initialized: now 1786346103034477 us; error 29 us; skew 500 ppm
Aug 10 07:15:03 dist-test-slave-10pc krb5kdc[20113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346103, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:03.037261 20375 init.cc:377] Logged in from keytab as kudu/127.15.217.66@KRBTEST.COM (short username kudu)
I20260810 07:15:03.037778 20375 webserver.cc:460] Webserver started at http://127.15.217.66:42111/ using document root <none> and password file <none>
I20260810 07:15:03.037993 20375 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:03.038041 20375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:03.038141 20375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:03.039219 20375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-1/data/instance:
uuid: "6d3bc527ff8049b786cb6dc05fd4011b"
format_stamp: "Formatted at 2026-08-10 07:15:03 on dist-test-slave-10pc"
server_key: "d9322f3a95e109e420f3e0872820a48f"
server_key_iv: "44b10f048a790a20408bbab82ebb3d9a"
server_key_version: "encryptionkey@0"
I20260810 07:15:03.039553 20375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance:
uuid: "6d3bc527ff8049b786cb6dc05fd4011b"
format_stamp: "Formatted at 2026-08-10 07:15:03 on dist-test-slave-10pc"
server_key: "d9322f3a95e109e420f3e0872820a48f"
server_key_iv: "44b10f048a790a20408bbab82ebb3d9a"
server_key_version: "encryptionkey@0"
I20260810 07:15:03.041002 20375 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:15:03.041798 20390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:03.042014 20375 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:15:03.042122 20375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-1/wal
uuid: "6d3bc527ff8049b786cb6dc05fd4011b"
format_stamp: "Formatted at 2026-08-10 07:15:03 on dist-test-slave-10pc"
server_key: "d9322f3a95e109e420f3e0872820a48f"
server_key_iv: "44b10f048a790a20408bbab82ebb3d9a"
server_key_version: "encryptionkey@0"
I20260810 07:15:03.042214 20375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:03.064944 20375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:03.067485 20375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:03.067709 20375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:03.068046 20375 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:03.068427 20375 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:03.068528 20375 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:03.068588 20375 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:03.068627 20375 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:03.075464 20375 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.66:34739
I20260810 07:15:03.075541 20513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.66:34739 every 8 connection(s)
I20260810 07:15:03.075942 20375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
I20260810 07:15:03.082804 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 20375
I20260810 07:15:03.082906 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance
Aug 10 07:15:03 dist-test-slave-10pc krb5kdc[20113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346103, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:15:03.083106 16229 external_mini_cluster.cc:1428] Setting key f3180510bfcb23ce0ad9caad020a8ea5
I20260810 07:15:03.087162 20209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:03.076925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:60291 (local address 127.15.217.126:39469)
0810 07:15:03.076990 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:03.076992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:03.077245 (+ 253us) server_negotiation.cc:407] Connection header received
0810 07:15:03.078318 (+ 1073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:03.078320 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:03.078334 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:03.078397 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:03.079171 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:03.079722 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:03.080935 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:03.081146 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:03.081233 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:03.083697 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:03.083698 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:03.083699 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:03.083706 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:03.085443 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:03.086058 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:03.086059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:03.086060 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:03.086135 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:03.086405 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:03.086406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:03.086407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:03.086574 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:03.086652 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:03.086931 (+ 279us) server_negotiation.cc:299] Negotiation successful
0810 07:15:03.087029 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 07:15:03.087216 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:03.076888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:39469 (local address 127.15.217.66:60291)
0810 07:15:03.077098 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:03.077139 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:03.078186 (+ 1047us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:03.078482 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:03.078486 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:03.078782 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:03.079070 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:03.079076 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:03.079837 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:03.079838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:03.080835 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:03.080838 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:03.081248 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:03.081250 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:03.081390 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:03.081939 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:03.081949 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:03.083582 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:03.085544 (+ 1962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:03.085556 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:03.085559 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:03.085956 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:03.086213 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:03.086214 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:03.086214 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:03.086338 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:03.086645 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:03.086651 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:03.086790 (+ 139us) client_negotiation.cc:769] Sending connection context
0810 07:15:03.086880 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 07:15:03.087023 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":102,"threads_started":1}
I20260810 07:15:03.087797 20514 heartbeater.cc:347] Connected to a master server at 127.15.217.126:39469
I20260810 07:15:03.087924 20514 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:03.088152 20514 heartbeater.cc:511] Master 127.15.217.126:39469 requested a full tablet report, sending...
I20260810 07:15:03.088725 20160 ts_manager.cc:194] Registered new tserver with Master: 6d3bc527ff8049b786cb6dc05fd4011b (127.15.217.66:34739)
I20260810 07:15:03.089183 20160 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.66@KRBTEST.COM'} at 127.15.217.66:60291
WARNING: no policy specified for kudu/127.15.217.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab.
Entry for principal kudu/127.15.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab.
I20260810 07:15:03.115836 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.67:0
--local_ip_for_outbound_sockets=127.15.217.67
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:39469
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:33557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:03.205673 20519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:03.205940 20519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:03.205987 20519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:03.207259 20519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:03.207341 20519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:03.207391 20519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.67
I20260810 07:15:03.209087 20519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:33557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:39469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.67
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:03.209329 20519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:03.209533 20519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:03.215117 20524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:03.215137 20527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:03.215078 20525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:03.215458 20519 server_base.cc:1034] running on GCE node
I20260810 07:15:03.215652 20519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:03.215873 20519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:03.217015 20519 hybrid_clock.cc:648] HybridClock initialized: now 1786346103217004 us; error 32 us; skew 500 ppm
Aug 10 07:15:03 dist-test-slave-10pc krb5kdc[20113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346103, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:03.219650 20519 init.cc:377] Logged in from keytab as kudu/127.15.217.67@KRBTEST.COM (short username kudu)
I20260810 07:15:03.220162 20519 webserver.cc:460] Webserver started at http://127.15.217.67:34205/ using document root <none> and password file <none>
I20260810 07:15:03.220398 20519 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:03.220445 20519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:03.220643 20519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:03.221794 20519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-2/data/instance:
uuid: "87690461ab7e41b190df440e16dab754"
format_stamp: "Formatted at 2026-08-10 07:15:03 on dist-test-slave-10pc"
server_key: "f7c1d8c9bf796c81e5fd99bc8a7b91b0"
server_key_iv: "3673d165844f8211e910ae7170da245d"
server_key_version: "encryptionkey@0"
I20260810 07:15:03.222179 20519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance:
uuid: "87690461ab7e41b190df440e16dab754"
format_stamp: "Formatted at 2026-08-10 07:15:03 on dist-test-slave-10pc"
server_key: "f7c1d8c9bf796c81e5fd99bc8a7b91b0"
server_key_iv: "3673d165844f8211e910ae7170da245d"
server_key_version: "encryptionkey@0"
I20260810 07:15:03.223611 20519 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:15:03.224421 20534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:03.224646 20519 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:15:03.224748 20519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-2/wal
uuid: "87690461ab7e41b190df440e16dab754"
format_stamp: "Formatted at 2026-08-10 07:15:03 on dist-test-slave-10pc"
server_key: "f7c1d8c9bf796c81e5fd99bc8a7b91b0"
server_key_iv: "3673d165844f8211e910ae7170da245d"
server_key_version: "encryptionkey@0"
I20260810 07:15:03.224812 20519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:03.262867 20519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:03.265381 20519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:03.265506 20519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:03.265731 20519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:03.266098 20519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:03.266137 20519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:03.266199 20519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:03.266232 20519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:03.272872 20519 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.67:40397
I20260810 07:15:03.272960 20656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.67:40397 every 8 connection(s)
I20260810 07:15:03.273377 20519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
Aug 10 07:15:03 dist-test-slave-10pc krb5kdc[20113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346103, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:15:03.282961 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 20519
I20260810 07:15:03.283064 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance
I20260810 07:15:03.283291 16229 external_mini_cluster.cc:1428] Setting key ddebf2e3955346abcfd7b396a051bb9a
I20260810 07:15:03.285255 20209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:03.274285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:50387 (local address 127.15.217.126:39469)
0810 07:15:03.274487 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:03.274490 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:03.274546 (+ 56us) server_negotiation.cc:407] Connection header received
0810 07:15:03.276106 (+ 1560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:03.276108 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:03.276121 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:03.276185 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:03.276963 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:03.277688 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:03.278765 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:03.279021 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:03.279088 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:03.281580 (+ 2492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:03.281581 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:03.281582 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:03.281590 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:03.283415 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:03.284025 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:03.284027 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:03.284027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:03.284101 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:03.284424 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:03.284425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:03.284425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:03.284602 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:03.284666 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:03.285086 (+ 420us) server_negotiation.cc:299] Negotiation successful
0810 07:15:03.285184 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175}
I20260810 07:15:03.285519 20659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:03.274240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:39469 (local address 127.15.217.67:50387)
0810 07:15:03.274422 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:03.274467 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:03.275978 (+ 1511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:03.276277 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:03.276284 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:03.276636 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:03.276850 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:03.276855 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:03.277813 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:03.277814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:03.278665 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:03.278668 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:03.279098 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:03.279099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:03.279239 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:03.279863 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:03.279872 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:03.281468 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:03.283515 (+ 2047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:03.283524 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:03.283527 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:03.283909 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:03.284182 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:03.284183 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:03.284184 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:03.284353 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:03.284685 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:03.284695 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:03.284888 (+ 193us) client_negotiation.cc:769] Sending connection context
0810 07:15:03.285047 (+ 159us) client_negotiation.cc:240] Negotiation successful
0810 07:15:03.285231 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":89,"threads_started":1}
I20260810 07:15:03.286191 20657 heartbeater.cc:347] Connected to a master server at 127.15.217.126:39469
I20260810 07:15:03.286300 20657 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:03.286512 20657 heartbeater.cc:511] Master 127.15.217.126:39469 requested a full tablet report, sending...
I20260810 07:15:03.286979 20160 ts_manager.cc:194] Registered new tserver with Master: 87690461ab7e41b190df440e16dab754 (127.15.217.67:40397)
I20260810 07:15:03.287436 20160 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.67@KRBTEST.COM'} at 127.15.217.67:50387
I20260810 07:15:03.295248 16229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 07:15:03 dist-test-slave-10pc krb5kdc[20113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 07:15:03 dist-test-slave-10pc krb5kdc[20113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.65@KRBTEST.COM
I20260810 07:15:03.321039 20664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:03.312641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55282 (local address 127.15.217.65:39117)
0810 07:15:03.312847 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:03.312854 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:03.312873 (+ 19us) server_negotiation.cc:407] Connection header received
0810 07:15:03.312906 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:03.312908 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:03.312923 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:03.313034 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:03.313658 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:03.314507 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:03.315258 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:03.315628 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:03.315688 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:03.318286 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:03.318298 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:03.318298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:03.318310 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:03.319889 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:03.320211 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:03.320214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:03.320214 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:03.320277 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:03.320513 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:03.320514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:03.320514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:03.320664 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:03.320774 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:03.320844 (+ 70us) server_negotiation.cc:299] Negotiation successful
0810 07:15:03.320915 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":126,"threads_started":1}
Aug 10 07:15:03 dist-test-slave-10pc krb5kdc[20113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:15:03.329214 20209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:03.322472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39350 (local address 127.15.217.126:39469)
0810 07:15:03.322534 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:03.322536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:03.322544 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:15:03.322607 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:03.322608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:03.322617 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:03.322662 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:03.323094 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:03.323587 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:03.324284 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:03.324553 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:03.324606 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:03.326825 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:03.326826 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:03.326827 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:03.326851 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:03.328115 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:03.328445 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:03.328446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:03.328446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:03.328548 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:03.328715 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:03.328715 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:03.328716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:03.328858 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:03.328934 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:03.329018 (+ 84us) server_negotiation.cc:299] Negotiation successful
0810 07:15:03.329085 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260810 07:15:03.329517 20160 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39350
I20260810 07:15:03.329948 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 20231
I20260810 07:15:03.338951 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 20375
I20260810 07:15:03.349678 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 20519
I20260810 07:15:03.357254 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 20127
2026-08-10T07:15:03Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2011 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:15:03 dist-test-slave-10pc krb5kdc[20669](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:15:03 dist-test-slave-10pc krb5kdc[20669](info): set up 2 sockets
Aug 10 07:15:03 dist-test-slave-10pc krb5kdc[20669](info): commencing operation
krb5kdc: starting...
W20260810 07:15:04.473177 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:15:04 dist-test-slave-10pc krb5kdc[20669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346104, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:15:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:15:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
I20260810 07:15:04.601428 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:43113
--webserver_interface=127.15.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:43939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:43113
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:04.691316 20683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:04.691540 20683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:04.691562 20683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:04.692770 20683 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:04.692816 20683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:04.692831 20683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:04.692844 20683 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:04.692857 20683 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:15:04.694330 20683 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:43939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:43113
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:43113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:04.694541 20683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:04.694749 20683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:04.699782 20692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:04.699783 20690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:04.699777 20689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:04.699954 20683 server_base.cc:1034] running on GCE node
I20260810 07:15:04.700416 20683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:04.700821 20683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:04.702039 20683 hybrid_clock.cc:648] HybridClock initialized: now 1786346104702036 us; error 47 us; skew 500 ppm
Aug 10 07:15:04 dist-test-slave-10pc krb5kdc[20669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346104, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:04.704936 20683 init.cc:377] Logged in from keytab as kudu/127.15.217.126@KRBTEST.COM (short username kudu)
I20260810 07:15:04.705335 20683 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:04.705394 20683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:04.705492 20683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:04.706604 20683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "170460c06ef54054bcaf901baaa06408"
format_stamp: "Formatted at 2026-08-10 07:15:04 on dist-test-slave-10pc"
server_key: "b5d6907f32703df687a3d31610ca3073"
server_key_iv: "6b65af5cf726fcc16196643fd09bc765"
server_key_version: "encryptionkey@0"
I20260810 07:15:04.706941 20683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "170460c06ef54054bcaf901baaa06408"
format_stamp: "Formatted at 2026-08-10 07:15:04 on dist-test-slave-10pc"
server_key: "b5d6907f32703df687a3d31610ca3073"
server_key_iv: "6b65af5cf726fcc16196643fd09bc765"
server_key_version: "encryptionkey@0"
I20260810 07:15:04.708267 20683 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 07:15:04.709097 20699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:04.709323 20683 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 07:15:04.709436 20683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "170460c06ef54054bcaf901baaa06408"
format_stamp: "Formatted at 2026-08-10 07:15:04 on dist-test-slave-10pc"
server_key: "b5d6907f32703df687a3d31610ca3073"
server_key_iv: "6b65af5cf726fcc16196643fd09bc765"
server_key_version: "encryptionkey@0"
I20260810 07:15:04.709517 20683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:04.718034 20683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:04.720201 20683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:04.720392 20683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:04.724756 20683 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:43113
I20260810 07:15:04.724797 20755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:43113 every 8 connection(s)
I20260810 07:15:04.725149 20683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:15:04.726336 20756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:04.727411 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 20683
I20260810 07:15:04.727550 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:15:04.727794 16229 external_mini_cluster.cc:1428] Setting key 9ffcba55185a17dcad89f93c3ae01a59
I20260810 07:15:04.730414 20756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 170460c06ef54054bcaf901baaa06408: Bootstrap starting.
I20260810 07:15:04.731163 20756 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 170460c06ef54054bcaf901baaa06408: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:04.731576 20756 log.cc:824] T 00000000000000000000000000000000 P 170460c06ef54054bcaf901baaa06408: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:04.732527 20756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 170460c06ef54054bcaf901baaa06408: No bootstrap required, opened a new log
Aug 10 07:15:04 dist-test-slave-10pc krb5kdc[20669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346104, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:15:04.735532 20756 raft_consensus.cc:348] T 00000000000000000000000000000000 P 170460c06ef54054bcaf901baaa06408 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "170460c06ef54054bcaf901baaa06408" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 43113 } }
I20260810 07:15:04.735754 20756 raft_consensus.cc:374] T 00000000000000000000000000000000 P 170460c06ef54054bcaf901baaa06408 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:04.735785 20756 raft_consensus.cc:729] T 00000000000000000000000000000000 P 170460c06ef54054bcaf901baaa06408 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170460c06ef54054bcaf901baaa06408, State: Initialized, Role: FOLLOWER
I20260810 07:15:04.735958 20756 consensus_queue.cc:260] T 00000000000000000000000000000000 P 170460c06ef54054bcaf901baaa06408 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "170460c06ef54054bcaf901baaa06408" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 43113 } }
I20260810 07:15:04.736034 20756 raft_consensus.cc:388] T 00000000000000000000000000000000 P 170460c06ef54054bcaf901baaa06408 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:15:04.736059 20756 raft_consensus.cc:482] T 00000000000000000000000000000000 P 170460c06ef54054bcaf901baaa06408 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:15:04.736171 20756 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 170460c06ef54054bcaf901baaa06408 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:04.737069 20756 raft_consensus.cc:504] T 00000000000000000000000000000000 P 170460c06ef54054bcaf901baaa06408 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "170460c06ef54054bcaf901baaa06408" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 43113 } }
I20260810 07:15:04.737203 20756 leader_election.cc:302] T 00000000000000000000000000000000 P 170460c06ef54054bcaf901baaa06408 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 170460c06ef54054bcaf901baaa06408; no voters:
I20260810 07:15:04.737465 20756 leader_election.cc:288] T 00000000000000000000000000000000 P 170460c06ef54054bcaf901baaa06408 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:15:04.737491 20761 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 170460c06ef54054bcaf901baaa06408 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:04.737618 20761 raft_consensus.cc:686] T 00000000000000000000000000000000 P 170460c06ef54054bcaf901baaa06408 [term 1 LEADER]: Becoming Leader. State: Replica: 170460c06ef54054bcaf901baaa06408, State: Running, Role: LEADER
I20260810 07:15:04.737793 20756 sys_catalog.cc:564] T 00000000000000000000000000000000 P 170460c06ef54054bcaf901baaa06408 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:04.737782 20761 consensus_queue.cc:237] T 00000000000000000000000000000000 P 170460c06ef54054bcaf901baaa06408 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "170460c06ef54054bcaf901baaa06408" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 43113 } }
I20260810 07:15:04.738363 20763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 170460c06ef54054bcaf901baaa06408 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 170460c06ef54054bcaf901baaa06408. Latest consensus state: current_term: 1 leader_uuid: "170460c06ef54054bcaf901baaa06408" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "170460c06ef54054bcaf901baaa06408" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 43113 } } }
I20260810 07:15:04.738466 20763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 170460c06ef54054bcaf901baaa06408 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:04.738732 20762 sys_catalog.cc:455] T 00000000000000000000000000000000 P 170460c06ef54054bcaf901baaa06408 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "170460c06ef54054bcaf901baaa06408" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "170460c06ef54054bcaf901baaa06408" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 43113 } } }
I20260810 07:15:04.739038 20768 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:04.739766 20768 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:15:04.740365 20759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:04.728417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50012 (local address 127.15.217.126:43113)
0810 07:15:04.728639 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:04.728648 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:04.728687 (+ 39us) server_negotiation.cc:407] Connection header received
0810 07:15:04.729161 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:04.729171 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:04.729221 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:04.729348 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:04.730359 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.731371 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.732566 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.733047 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.733135 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:04.736181 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:04.736185 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:04.736185 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:04.736198 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:04.738217 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:04.738716 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:04.738720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:04.738720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:04.738869 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:04.739112 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:04.739113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:04.739114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:04.739466 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:04.739630 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:04.740009 (+ 379us) server_negotiation.cc:299] Negotiation successful
0810 07:15:04.740181 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":134,"threads_started":1}
I20260810 07:15:04.741304 20762 sys_catalog.cc:458] T 00000000000000000000000000000000 P 170460c06ef54054bcaf901baaa06408 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:04.741647 20768 catalog_manager.cc:1347] Generated new cluster ID: 916e5e5e92c54f228452888527062460
I20260810 07:15:04.741700 20768 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:15:04.762307 20768 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:15:04.763270 20768 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:04.770105 20768 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 170460c06ef54054bcaf901baaa06408: Generated new TSK 0
I20260810 07:15:04.770370 20768 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab.
Entry for principal kudu/127.15.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab.
I20260810 07:15:04.800320 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.65:0
--local_ip_for_outbound_sockets=127.15.217.65
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:43113
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:43939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:04.887275 20784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:04.887547 20784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:04.887598 20784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:04.888907 20784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:04.888983 20784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:04.889030 20784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.65
I20260810 07:15:04.890667 20784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:43939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:43113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.65
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:04.890946 20784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:04.891201 20784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:04.896431 20790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:04.896499 20784 server_base.cc:1034] running on GCE node
W20260810 07:15:04.896412 20794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:04.896413 20791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:04.896965 20784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:04.897223 20784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:04.898403 20784 hybrid_clock.cc:648] HybridClock initialized: now 1786346104898383 us; error 37 us; skew 500 ppm
Aug 10 07:15:04 dist-test-slave-10pc krb5kdc[20669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346104, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:04.901335 20784 init.cc:377] Logged in from keytab as kudu/127.15.217.65@KRBTEST.COM (short username kudu)
I20260810 07:15:04.901713 20784 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:04.901795 20784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:04.901944 20784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:04.903048 20784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-0/data/instance:
uuid: "bf39e1fb29a747089147b7cb4af631c2"
format_stamp: "Formatted at 2026-08-10 07:15:04 on dist-test-slave-10pc"
server_key: "bc5b0b18011d104e85916757b370d439"
server_key_iv: "1e10d8c00fd9d2e0a254f44ecd3f9d07"
server_key_version: "encryptionkey@0"
I20260810 07:15:04.903429 20784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance:
uuid: "bf39e1fb29a747089147b7cb4af631c2"
format_stamp: "Formatted at 2026-08-10 07:15:04 on dist-test-slave-10pc"
server_key: "bc5b0b18011d104e85916757b370d439"
server_key_iv: "1e10d8c00fd9d2e0a254f44ecd3f9d07"
server_key_version: "encryptionkey@0"
I20260810 07:15:04.904820 20784 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:15:04.905706 20801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:04.905891 20784 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 07:15:04.906009 20784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-0/wal
uuid: "bf39e1fb29a747089147b7cb4af631c2"
format_stamp: "Formatted at 2026-08-10 07:15:04 on dist-test-slave-10pc"
server_key: "bc5b0b18011d104e85916757b370d439"
server_key_iv: "1e10d8c00fd9d2e0a254f44ecd3f9d07"
server_key_version: "encryptionkey@0"
I20260810 07:15:04.906111 20784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:04.931809 20784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:04.934026 20784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:04.934224 20784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:04.934531 20784 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:04.934906 20784 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:04.934971 20784 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:04.935011 20784 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:04.935047 20784 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:04.941721 20784 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.65:46347
I20260810 07:15:04.941766 20916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.65:46347 every 8 connection(s)
I20260810 07:15:04.942101 20784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
I20260810 07:15:04.947422 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 20784
I20260810 07:15:04.947546 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance
I20260810 07:15:04.947763 16229 external_mini_cluster.cc:1428] Setting key 967121322b373a64afbb4d7d995afe13
Aug 10 07:15:04 dist-test-slave-10pc krb5kdc[20669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346104, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:15:04.953030 20759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:04.942978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:34589 (local address 127.15.217.126:43113)
0810 07:15:04.943045 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:04.943047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:04.943245 (+ 198us) server_negotiation.cc:407] Connection header received
0810 07:15:04.943748 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:04.943750 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:04.943760 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:04.943826 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:04.944869 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.945465 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.946541 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.946788 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.946855 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:04.949634 (+ 2779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:04.949636 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:04.949637 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:04.949645 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:04.951186 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:04.951766 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:04.951768 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:04.951768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:04.951825 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:04.952178 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:04.952179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:04.952180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:04.952344 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:04.952419 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:04.952778 (+ 359us) server_negotiation.cc:299] Negotiation successful
0810 07:15:04.952886 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 07:15:04.953121 20919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:04.942929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43113 (local address 127.15.217.65:34589)
0810 07:15:04.943111 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:04.943156 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:04.943651 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:04.943906 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:04.943909 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:04.944189 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:04.944773 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:04.944780 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.945587 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.945588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:04.946426 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:04.946429 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.946878 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.946879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:04.946992 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:04.947570 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:04.947582 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:04.949524 (+ 1942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:04.951290 (+ 1766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:04.951300 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:04.951304 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:04.951682 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:04.951893 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:04.951894 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:04.951894 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:04.952089 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:04.952419 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:04.952426 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:04.952626 (+ 200us) client_negotiation.cc:769] Sending connection context
0810 07:15:04.952727 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 07:15:04.952869 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":99,"threads_started":1}
I20260810 07:15:04.953790 20917 heartbeater.cc:347] Connected to a master server at 127.15.217.126:43113
I20260810 07:15:04.953944 20917 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:04.954154 20917 heartbeater.cc:511] Master 127.15.217.126:43113 requested a full tablet report, sending...
I20260810 07:15:04.954862 20719 ts_manager.cc:194] Registered new tserver with Master: bf39e1fb29a747089147b7cb4af631c2 (127.15.217.65:46347)
I20260810 07:15:04.956018 20719 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.65@KRBTEST.COM'} at 127.15.217.65:34589
WARNING: no policy specified for kudu/127.15.217.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab.
Entry for principal kudu/127.15.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab.
I20260810 07:15:04.976094 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.66:0
--local_ip_for_outbound_sockets=127.15.217.66
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:43113
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:43939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:05.063632 20923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:05.063854 20923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:05.063880 20923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:05.065125 20923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:05.065171 20923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:05.065227 20923 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.66
I20260810 07:15:05.066781 20923 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:43939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:43113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.66
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:05.067047 20923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:05.067305 20923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:05.072753 20930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:05.072795 20931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:05.072781 20923 server_base.cc:1034] running on GCE node
W20260810 07:15:05.072753 20933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:05.073252 20923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:05.073630 20923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:05.074812 20923 hybrid_clock.cc:648] HybridClock initialized: now 1786346105074796 us; error 31 us; skew 500 ppm
Aug 10 07:15:05 dist-test-slave-10pc krb5kdc[20669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346105, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:05.077659 20923 init.cc:377] Logged in from keytab as kudu/127.15.217.66@KRBTEST.COM (short username kudu)
I20260810 07:15:05.078083 20923 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:05.078164 20923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:05.078321 20923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:05.079663 20923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-1/data/instance:
uuid: "cb0c828730d04c65a1bca1543e765658"
format_stamp: "Formatted at 2026-08-10 07:15:05 on dist-test-slave-10pc"
server_key: "1e6fed02a81188c5165d3d6fa89d5fc5"
server_key_iv: "980a71d799b4e06b4711f54c2845104e"
server_key_version: "encryptionkey@0"
I20260810 07:15:05.080154 20923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance:
uuid: "cb0c828730d04c65a1bca1543e765658"
format_stamp: "Formatted at 2026-08-10 07:15:05 on dist-test-slave-10pc"
server_key: "1e6fed02a81188c5165d3d6fa89d5fc5"
server_key_iv: "980a71d799b4e06b4711f54c2845104e"
server_key_version: "encryptionkey@0"
I20260810 07:15:05.081595 20923 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:15:05.082448 20940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:05.082631 20923 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 07:15:05.082742 20923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-1/wal
uuid: "cb0c828730d04c65a1bca1543e765658"
format_stamp: "Formatted at 2026-08-10 07:15:05 on dist-test-slave-10pc"
server_key: "1e6fed02a81188c5165d3d6fa89d5fc5"
server_key_iv: "980a71d799b4e06b4711f54c2845104e"
server_key_version: "encryptionkey@0"
I20260810 07:15:05.082821 20923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:05.098256 20923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:05.100270 20923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:05.100560 20923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:05.100836 20923 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:05.101188 20923 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:05.101244 20923 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:05.101289 20923 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:05.101315 20923 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:05.107506 20923 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.66:44807
I20260810 07:15:05.107547 21058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.66:44807 every 8 connection(s)
I20260810 07:15:05.107824 20923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
I20260810 07:15:05.112684 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 20923
I20260810 07:15:05.112862 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance
I20260810 07:15:05.113112 16229 external_mini_cluster.cc:1428] Setting key 3445c728823ba2ef3c77174582b775ef
Aug 10 07:15:05 dist-test-slave-10pc krb5kdc[20669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346105, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:15:05.118163 20759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:05.108572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:36605 (local address 127.15.217.126:43113)
0810 07:15:05.108648 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:05.108651 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:05.108853 (+ 202us) server_negotiation.cc:407] Connection header received
0810 07:15:05.109464 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:05.109472 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:05.109487 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:05.109547 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:05.110554 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.111066 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.112093 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.112288 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.112345 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:05.115005 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:05.115007 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:05.115007 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:05.115016 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:05.116567 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:05.117072 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:05.117072 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:05.117073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:05.117131 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:05.117420 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:05.117421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:05.117422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:05.117574 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:05.117646 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:05.117911 (+ 265us) server_negotiation.cc:299] Negotiation successful
0810 07:15:05.118021 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 07:15:05.118188 21061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:05.108555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43113 (local address 127.15.217.66:36605)
0810 07:15:05.108709 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:05.108756 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:05.109338 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:05.109626 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:05.109630 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:05.109904 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:05.110439 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:05.110445 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.111160 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.111161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:05.112022 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:05.112024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.112366 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.112367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:05.112536 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:05.113115 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:05.113125 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:05.114906 (+ 1781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:05.116647 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:05.116656 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:05.116658 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:05.116991 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:05.117198 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:05.117199 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:05.117200 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:05.117349 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:05.117642 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:05.117648 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:05.117774 (+ 126us) client_negotiation.cc:769] Sending connection context
0810 07:15:05.117860 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 07:15:05.118002 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":73,"threads_started":1}
I20260810 07:15:05.118950 21059 heartbeater.cc:347] Connected to a master server at 127.15.217.126:43113
I20260810 07:15:05.119092 21059 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:05.119289 21059 heartbeater.cc:511] Master 127.15.217.126:43113 requested a full tablet report, sending...
I20260810 07:15:05.119774 20719 ts_manager.cc:194] Registered new tserver with Master: cb0c828730d04c65a1bca1543e765658 (127.15.217.66:44807)
I20260810 07:15:05.120333 20719 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.66@KRBTEST.COM'} at 127.15.217.66:36605
WARNING: no policy specified for kudu/127.15.217.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab.
Entry for principal kudu/127.15.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab.
I20260810 07:15:05.144532 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.67:0
--local_ip_for_outbound_sockets=127.15.217.67
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:43113
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:43939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:05.232905 21065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:05.233122 21065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:05.233147 21065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:05.234331 21065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:05.234372 21065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:05.234400 21065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.67
I20260810 07:15:05.235849 21065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:43939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:43113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.67
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:05.236052 21065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:05.236232 21065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:05.241711 21075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:05.241725 21072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:05.241735 21070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:05.241987 21065 server_base.cc:1034] running on GCE node
I20260810 07:15:05.242195 21065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:05.242403 21065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:05.243551 21065 hybrid_clock.cc:648] HybridClock initialized: now 1786346105243535 us; error 29 us; skew 500 ppm
Aug 10 07:15:05 dist-test-slave-10pc krb5kdc[20669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346105, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:05.246327 21065 init.cc:377] Logged in from keytab as kudu/127.15.217.67@KRBTEST.COM (short username kudu)
I20260810 07:15:05.246666 21065 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:05.246716 21065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:05.246809 21065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:05.247845 21065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-2/data/instance:
uuid: "cb587703ffb2471cb304f6d624d407b9"
format_stamp: "Formatted at 2026-08-10 07:15:05 on dist-test-slave-10pc"
server_key: "7b4c06907fc29814351bef6d7322928f"
server_key_iv: "b4bcc1c0a52bbe1249ea5f78bc337e03"
server_key_version: "encryptionkey@0"
I20260810 07:15:05.248167 21065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance:
uuid: "cb587703ffb2471cb304f6d624d407b9"
format_stamp: "Formatted at 2026-08-10 07:15:05 on dist-test-slave-10pc"
server_key: "7b4c06907fc29814351bef6d7322928f"
server_key_iv: "b4bcc1c0a52bbe1249ea5f78bc337e03"
server_key_version: "encryptionkey@0"
I20260810 07:15:05.249543 21065 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:15:05.250285 21081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:05.250500 21065 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:05.250579 21065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-2/wal
uuid: "cb587703ffb2471cb304f6d624d407b9"
format_stamp: "Formatted at 2026-08-10 07:15:05 on dist-test-slave-10pc"
server_key: "7b4c06907fc29814351bef6d7322928f"
server_key_iv: "b4bcc1c0a52bbe1249ea5f78bc337e03"
server_key_version: "encryptionkey@0"
I20260810 07:15:05.250635 21065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:05.263641 21065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:05.265894 21065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:05.266033 21065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:05.266243 21065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:05.266577 21065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:05.266613 21065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:05.266656 21065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:05.266683 21065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:05.273185 21065 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.67:46285
I20260810 07:15:05.273221 21204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.67:46285 every 8 connection(s)
I20260810 07:15:05.273517 21065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
Aug 10 07:15:05 dist-test-slave-10pc krb5kdc[20669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346105, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:15:05.280889 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 21065
I20260810 07:15:05.281265 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance
I20260810 07:15:05.281544 16229 external_mini_cluster.cc:1428] Setting key 51662cba55e8b23e1f31c5475908b8a5
I20260810 07:15:05.283278 20759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:05.274294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:33319 (local address 127.15.217.126:43113)
0810 07:15:05.274361 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:05.274363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:05.274617 (+ 254us) server_negotiation.cc:407] Connection header received
0810 07:15:05.275138 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:05.275140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:05.275151 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:05.275206 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:05.276186 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.276768 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.277753 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.277940 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.278011 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:05.280100 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:05.280102 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:05.280102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:05.280109 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:05.281875 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:05.282375 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:05.282376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:05.282377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:05.282442 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:05.282737 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:05.282737 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:05.282738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:05.282845 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:05.282899 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:05.283148 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 07:15:05.283213 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 07:15:05.283478 21207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:05.274272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43113 (local address 127.15.217.67:33319)
0810 07:15:05.274437 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:05.274476 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:05.275038 (+ 562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:05.275286 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:05.275290 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:05.275572 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:05.276092 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:05.276097 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.276849 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.276850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:05.277673 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:05.277676 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.278040 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.278041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:05.278148 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:05.278601 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:05.278610 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:05.280009 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:05.281968 (+ 1959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:05.281975 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:05.281977 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:05.282301 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:05.282546 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:05.282547 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:05.282547 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:05.282673 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:05.282906 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:05.282912 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:05.283031 (+ 119us) client_negotiation.cc:769] Sending connection context
0810 07:15:05.283116 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 07:15:05.283237 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":95,"threads_started":1}
I20260810 07:15:05.283975 21205 heartbeater.cc:347] Connected to a master server at 127.15.217.126:43113
I20260810 07:15:05.284085 21205 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:05.284243 21205 heartbeater.cc:511] Master 127.15.217.126:43113 requested a full tablet report, sending...
I20260810 07:15:05.284673 20719 ts_manager.cc:194] Registered new tserver with Master: cb587703ffb2471cb304f6d624d407b9 (127.15.217.67:46285)
I20260810 07:15:05.285024 20719 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.67@KRBTEST.COM'} at 127.15.217.67:33319
I20260810 07:15:05.293371 16229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:15:05.300921 20759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:05.294673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50026 (local address 127.15.217.126:43113)
0810 07:15:05.294744 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:05.294747 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:05.294804 (+ 57us) server_negotiation.cc:407] Connection header received
0810 07:15:05.294859 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:05.294859 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:05.294868 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:05.294913 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:05.295466 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.295943 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.296783 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.297038 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.297096 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:05.298202 (+ 1106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:05.298203 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:05.298203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:05.298209 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:05.299630 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:05.299983 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:05.299984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:05.299984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:05.300045 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:05.300340 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:05.300341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:05.300341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:05.300584 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:05.300683 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:05.300766 (+ 83us) server_negotiation.cc:299] Negotiation successful
0810 07:15:05.300830 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:15:05.302599 20719 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50026:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:15:05.304270 20719 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:15:05.311514 21222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:05.307340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44354 (local address 127.15.217.65:46347)
0810 07:15:05.307516 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:05.307522 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:05.307534 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:15:05.307572 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:05.307574 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:05.307590 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:05.307832 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:05.307920 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.309038 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.310284 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.311072 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.311153 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:05.311217 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:05.311383 (+ 166us) server_negotiation.cc:299] Negotiation successful
0810 07:15:05.311425 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":106,"threads_started":1}
I20260810 07:15:05.311528 21221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:05.307079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:46347 (local address 127.0.0.1:44354)
0810 07:15:05.307220 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:05.307231 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:05.307286 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:05.307700 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:05.307701 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:05.307703 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:05.307748 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:05.307750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.309175 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.309177 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:05.310195 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:05.310198 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.311164 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.311165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:05.311255 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:05.311274 (+ 19us) client_negotiation.cc:769] Sending connection context
0810 07:15:05.311357 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 07:15:05.311428 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":95,"threads_started":1}
I20260810 07:15:05.311813 21219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:05.306980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58638 (local address 127.15.217.66:44807)
0810 07:15:05.307391 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:05.307395 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:05.307404 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:15:05.307444 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:05.307446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:05.307461 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:05.307549 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:05.307765 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.308987 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.310372 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.311071 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.311153 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:05.311217 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:05.311681 (+ 464us) server_negotiation.cc:299] Negotiation successful
0810 07:15:05.311733 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":76,"threads_started":1}
I20260810 07:15:05.311528 21217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:05.306817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:44807 (local address 127.0.0.1:58638)
0810 07:15:05.307026 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:05.307073 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:05.307143 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:05.307565 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:05.307569 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:05.307572 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:05.307668 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:05.307671 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.309097 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.309098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:05.310285 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:05.310287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.311173 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.311174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:05.311255 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:05.311274 (+ 19us) client_negotiation.cc:769] Sending connection context
0810 07:15:05.311358 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 07:15:05.311428 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":61,"thread_start_us":70,"threads_started":1}
I20260810 07:15:05.312393 20851 tablet_service.cc:1467] Processing CreateTablet for tablet d5e40dacbb7c4670844320aa2ba2a457 (DEFAULT_TABLE table=test-table [id=9d6008c7effd4fd385c8417785c0707c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:05.312572 21216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:05.306864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:46285 (local address 127.0.0.1:48512)
0810 07:15:05.307193 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:05.307203 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:05.307249 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:05.307578 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:05.307579 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:05.307581 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:05.307696 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:05.307699 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.308776 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.308777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:05.310284 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:05.310287 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.312283 (+ 1996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.312284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:05.312355 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:05.312360 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:15:05.312411 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 07:15:05.312494 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"mutex_wait_us":25,"thread_start_us":103,"threads_started":1}
I20260810 07:15:05.312881 20851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5e40dacbb7c4670844320aa2ba2a457. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:05.312902 21220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:05.306997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48512 (local address 127.15.217.67:46285)
0810 07:15:05.307307 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:05.307311 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:05.307324 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:15:05.307371 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:05.307373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:05.307391 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:05.307467 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:05.307796 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.308657 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.311258 (+ 2601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.312117 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.312184 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:05.312248 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:05.312771 (+ 523us) server_negotiation.cc:299] Negotiation successful
0810 07:15:05.312816 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":101,"threads_started":1}
I20260810 07:15:05.312990 20990 tablet_service.cc:1467] Processing CreateTablet for tablet d5e40dacbb7c4670844320aa2ba2a457 (DEFAULT_TABLE table=test-table [id=9d6008c7effd4fd385c8417785c0707c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:05.313319 20990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5e40dacbb7c4670844320aa2ba2a457. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:05.313571 21133 tablet_service.cc:1467] Processing CreateTablet for tablet d5e40dacbb7c4670844320aa2ba2a457 (DEFAULT_TABLE table=test-table [id=9d6008c7effd4fd385c8417785c0707c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:05.313838 21133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5e40dacbb7c4670844320aa2ba2a457. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:05.316380 21223 tablet_bootstrap.cc:492] T d5e40dacbb7c4670844320aa2ba2a457 P cb587703ffb2471cb304f6d624d407b9: Bootstrap starting.
I20260810 07:15:05.316730 21225 tablet_bootstrap.cc:492] T d5e40dacbb7c4670844320aa2ba2a457 P cb0c828730d04c65a1bca1543e765658: Bootstrap starting.
I20260810 07:15:05.317147 21223 tablet_bootstrap.cc:654] T d5e40dacbb7c4670844320aa2ba2a457 P cb587703ffb2471cb304f6d624d407b9: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:05.317405 21225 tablet_bootstrap.cc:654] T d5e40dacbb7c4670844320aa2ba2a457 P cb0c828730d04c65a1bca1543e765658: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:05.317634 21223 log.cc:824] T d5e40dacbb7c4670844320aa2ba2a457 P cb587703ffb2471cb304f6d624d407b9: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:05.317998 21225 log.cc:824] T d5e40dacbb7c4670844320aa2ba2a457 P cb0c828730d04c65a1bca1543e765658: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:05.318351 21224 tablet_bootstrap.cc:492] T d5e40dacbb7c4670844320aa2ba2a457 P bf39e1fb29a747089147b7cb4af631c2: Bootstrap starting.
I20260810 07:15:05.318413 21223 tablet_bootstrap.cc:492] T d5e40dacbb7c4670844320aa2ba2a457 P cb587703ffb2471cb304f6d624d407b9: No bootstrap required, opened a new log
I20260810 07:15:05.318545 21223 ts_tablet_manager.cc:1397] T d5e40dacbb7c4670844320aa2ba2a457 P cb587703ffb2471cb304f6d624d407b9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:05.318804 21225 tablet_bootstrap.cc:492] T d5e40dacbb7c4670844320aa2ba2a457 P cb0c828730d04c65a1bca1543e765658: No bootstrap required, opened a new log
I20260810 07:15:05.318859 21225 ts_tablet_manager.cc:1397] T d5e40dacbb7c4670844320aa2ba2a457 P cb0c828730d04c65a1bca1543e765658: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:05.320016 21223 raft_consensus.cc:348] T d5e40dacbb7c4670844320aa2ba2a457 P cb587703ffb2471cb304f6d624d407b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb0c828730d04c65a1bca1543e765658" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44807 } } peers { permanent_uuid: "cb587703ffb2471cb304f6d624d407b9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 46285 } } peers { permanent_uuid: "bf39e1fb29a747089147b7cb4af631c2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 46347 } }
I20260810 07:15:05.320181 21223 raft_consensus.cc:374] T d5e40dacbb7c4670844320aa2ba2a457 P cb587703ffb2471cb304f6d624d407b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:05.320228 21223 raft_consensus.cc:729] T d5e40dacbb7c4670844320aa2ba2a457 P cb587703ffb2471cb304f6d624d407b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb587703ffb2471cb304f6d624d407b9, State: Initialized, Role: FOLLOWER
I20260810 07:15:05.320289 21225 raft_consensus.cc:348] T d5e40dacbb7c4670844320aa2ba2a457 P cb0c828730d04c65a1bca1543e765658 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb0c828730d04c65a1bca1543e765658" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44807 } } peers { permanent_uuid: "cb587703ffb2471cb304f6d624d407b9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 46285 } } peers { permanent_uuid: "bf39e1fb29a747089147b7cb4af631c2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 46347 } }
I20260810 07:15:05.320412 21223 consensus_queue.cc:260] T d5e40dacbb7c4670844320aa2ba2a457 P cb587703ffb2471cb304f6d624d407b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb0c828730d04c65a1bca1543e765658" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44807 } } peers { permanent_uuid: "cb587703ffb2471cb304f6d624d407b9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 46285 } } peers { permanent_uuid: "bf39e1fb29a747089147b7cb4af631c2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 46347 } }
I20260810 07:15:05.320498 21225 raft_consensus.cc:374] T d5e40dacbb7c4670844320aa2ba2a457 P cb0c828730d04c65a1bca1543e765658 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:05.320549 21225 raft_consensus.cc:729] T d5e40dacbb7c4670844320aa2ba2a457 P cb0c828730d04c65a1bca1543e765658 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0c828730d04c65a1bca1543e765658, State: Initialized, Role: FOLLOWER
I20260810 07:15:05.320668 21225 consensus_queue.cc:260] T d5e40dacbb7c4670844320aa2ba2a457 P cb0c828730d04c65a1bca1543e765658 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb0c828730d04c65a1bca1543e765658" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44807 } } peers { permanent_uuid: "cb587703ffb2471cb304f6d624d407b9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 46285 } } peers { permanent_uuid: "bf39e1fb29a747089147b7cb4af631c2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 46347 } }
I20260810 07:15:05.320900 21223 ts_tablet_manager.cc:1428] T d5e40dacbb7c4670844320aa2ba2a457 P cb587703ffb2471cb304f6d624d407b9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:05.320974 21225 ts_tablet_manager.cc:1428] T d5e40dacbb7c4670844320aa2ba2a457 P cb0c828730d04c65a1bca1543e765658: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 07:15:05.320986 21205 heartbeater.cc:503] Master 127.15.217.126:43113 was elected leader, sending a full tablet report...
I20260810 07:15:05.320997 21059 heartbeater.cc:503] Master 127.15.217.126:43113 was elected leader, sending a full tablet report...
I20260810 07:15:05.321739 21224 tablet_bootstrap.cc:654] T d5e40dacbb7c4670844320aa2ba2a457 P bf39e1fb29a747089147b7cb4af631c2: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:05.322494 21224 log.cc:824] T d5e40dacbb7c4670844320aa2ba2a457 P bf39e1fb29a747089147b7cb4af631c2: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:05.323230 21224 tablet_bootstrap.cc:492] T d5e40dacbb7c4670844320aa2ba2a457 P bf39e1fb29a747089147b7cb4af631c2: No bootstrap required, opened a new log
I20260810 07:15:05.323287 21224 ts_tablet_manager.cc:1397] T d5e40dacbb7c4670844320aa2ba2a457 P bf39e1fb29a747089147b7cb4af631c2: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260810 07:15:05.324971 21224 raft_consensus.cc:348] T d5e40dacbb7c4670844320aa2ba2a457 P bf39e1fb29a747089147b7cb4af631c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb0c828730d04c65a1bca1543e765658" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44807 } } peers { permanent_uuid: "cb587703ffb2471cb304f6d624d407b9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 46285 } } peers { permanent_uuid: "bf39e1fb29a747089147b7cb4af631c2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 46347 } }
I20260810 07:15:05.325117 21224 raft_consensus.cc:374] T d5e40dacbb7c4670844320aa2ba2a457 P bf39e1fb29a747089147b7cb4af631c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:05.325141 21224 raft_consensus.cc:729] T d5e40dacbb7c4670844320aa2ba2a457 P bf39e1fb29a747089147b7cb4af631c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf39e1fb29a747089147b7cb4af631c2, State: Initialized, Role: FOLLOWER
I20260810 07:15:05.325268 21224 consensus_queue.cc:260] T d5e40dacbb7c4670844320aa2ba2a457 P bf39e1fb29a747089147b7cb4af631c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb0c828730d04c65a1bca1543e765658" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44807 } } peers { permanent_uuid: "cb587703ffb2471cb304f6d624d407b9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 46285 } } peers { permanent_uuid: "bf39e1fb29a747089147b7cb4af631c2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 46347 } }
I20260810 07:15:05.325556 21224 ts_tablet_manager.cc:1428] T d5e40dacbb7c4670844320aa2ba2a457 P bf39e1fb29a747089147b7cb4af631c2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:05.325577 20917 heartbeater.cc:503] Master 127.15.217.126:43113 was elected leader, sending a full tablet report...
W20260810 07:15:05.358387 21060 tablet.cc:2369] T d5e40dacbb7c4670844320aa2ba2a457 P cb0c828730d04c65a1bca1543e765658: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:15:05.442890 20918 tablet.cc:2369] T d5e40dacbb7c4670844320aa2ba2a457 P bf39e1fb29a747089147b7cb4af631c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:15:05.524197 21206 tablet.cc:2369] T d5e40dacbb7c4670844320aa2ba2a457 P cb587703ffb2471cb304f6d624d407b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:15:05.761873 21231 raft_consensus.cc:482] T d5e40dacbb7c4670844320aa2ba2a457 P cb0c828730d04c65a1bca1543e765658 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:15:05.762065 21231 raft_consensus.cc:504] T d5e40dacbb7c4670844320aa2ba2a457 P cb0c828730d04c65a1bca1543e765658 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb0c828730d04c65a1bca1543e765658" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44807 } } peers { permanent_uuid: "cb587703ffb2471cb304f6d624d407b9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 46285 } } peers { permanent_uuid: "bf39e1fb29a747089147b7cb4af631c2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 46347 } }
I20260810 07:15:05.762579 21231 leader_election.cc:288] T d5e40dacbb7c4670844320aa2ba2a457 P cb0c828730d04c65a1bca1543e765658 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb587703ffb2471cb304f6d624d407b9 (127.15.217.67:46285), bf39e1fb29a747089147b7cb4af631c2 (127.15.217.65:46347)
I20260810 07:15:05.765861 21222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:05.763160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:56489 (local address 127.15.217.65:46347)
0810 07:15:05.763222 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:05.763224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:05.763235 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:15:05.763316 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:05.763317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:05.763331 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:05.763398 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:05.763561 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.764036 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.765137 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.765506 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.765596 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:05.765606 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:05.765747 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 07:15:05.765792 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 07:15:05.765913 21220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:05.763021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:46175 (local address 127.15.217.67:46285)
0810 07:15:05.763088 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:05.763094 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:05.763109 (+ 15us) server_negotiation.cc:407] Connection header received
0810 07:15:05.763260 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:05.763261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:05.763280 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:05.763352 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:05.763528 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.764034 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.765130 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.765509 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.765601 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:05.765652 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:05.765798 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 07:15:05.765839 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 07:15:05.765951 21235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:05.763050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:46347 (local address 127.15.217.66:56489)
0810 07:15:05.763165 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:05.763175 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:05.763237 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:05.763402 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:05.763403 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:05.763405 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:05.763486 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:05.763488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.764123 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.764124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:05.765040 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:05.765042 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.765596 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.765597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:05.765670 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:05.765673 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:15:05.765738 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 07:15:05.765783 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":81,"threads_started":1}
I20260810 07:15:05.765977 21234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:05.762835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:46285 (local address 127.15.217.66:46175)
0810 07:15:05.762996 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:05.763008 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:05.763112 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:05.763358 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:05.763360 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:05.763363 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:05.763452 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:05.763454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.764127 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.764128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:05.765029 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:05.765032 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.765603 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.765604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:05.765717 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:05.765721 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:15:05.765790 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 07:15:05.765837 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":1,"thread_start_us":89,"threads_started":1}
I20260810 07:15:05.766405 20871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5e40dacbb7c4670844320aa2ba2a457" candidate_uuid: "cb0c828730d04c65a1bca1543e765658" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf39e1fb29a747089147b7cb4af631c2" is_pre_election: true
I20260810 07:15:05.766424 21154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5e40dacbb7c4670844320aa2ba2a457" candidate_uuid: "cb0c828730d04c65a1bca1543e765658" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb587703ffb2471cb304f6d624d407b9" is_pre_election: true
I20260810 07:15:05.766599 20871 raft_consensus.cc:2454] T d5e40dacbb7c4670844320aa2ba2a457 P bf39e1fb29a747089147b7cb4af631c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0c828730d04c65a1bca1543e765658 in term 0.
I20260810 07:15:05.766616 21154 raft_consensus.cc:2454] T d5e40dacbb7c4670844320aa2ba2a457 P cb587703ffb2471cb304f6d624d407b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0c828730d04c65a1bca1543e765658 in term 0.
I20260810 07:15:05.766875 20944 leader_election.cc:302] T d5e40dacbb7c4670844320aa2ba2a457 P cb0c828730d04c65a1bca1543e765658 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0c828730d04c65a1bca1543e765658, cb587703ffb2471cb304f6d624d407b9; no voters:
I20260810 07:15:05.767064 21231 raft_consensus.cc:2781] T d5e40dacbb7c4670844320aa2ba2a457 P cb0c828730d04c65a1bca1543e765658 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:15:05.767161 21231 raft_consensus.cc:482] T d5e40dacbb7c4670844320aa2ba2a457 P cb0c828730d04c65a1bca1543e765658 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:15:05.767185 21231 raft_consensus.cc:3037] T d5e40dacbb7c4670844320aa2ba2a457 P cb0c828730d04c65a1bca1543e765658 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:05.768234 21231 raft_consensus.cc:504] T d5e40dacbb7c4670844320aa2ba2a457 P cb0c828730d04c65a1bca1543e765658 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb0c828730d04c65a1bca1543e765658" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44807 } } peers { permanent_uuid: "cb587703ffb2471cb304f6d624d407b9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 46285 } } peers { permanent_uuid: "bf39e1fb29a747089147b7cb4af631c2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 46347 } }
I20260810 07:15:05.768404 21231 leader_election.cc:288] T d5e40dacbb7c4670844320aa2ba2a457 P cb0c828730d04c65a1bca1543e765658 [CANDIDATE]: Term 1 election: Requested vote from peers cb587703ffb2471cb304f6d624d407b9 (127.15.217.67:46285), bf39e1fb29a747089147b7cb4af631c2 (127.15.217.65:46347)
I20260810 07:15:05.768641 21154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5e40dacbb7c4670844320aa2ba2a457" candidate_uuid: "cb0c828730d04c65a1bca1543e765658" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb587703ffb2471cb304f6d624d407b9"
I20260810 07:15:05.768641 20871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5e40dacbb7c4670844320aa2ba2a457" candidate_uuid: "cb0c828730d04c65a1bca1543e765658" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf39e1fb29a747089147b7cb4af631c2"
I20260810 07:15:05.768738 20871 raft_consensus.cc:3037] T d5e40dacbb7c4670844320aa2ba2a457 P bf39e1fb29a747089147b7cb4af631c2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:05.768738 21154 raft_consensus.cc:3037] T d5e40dacbb7c4670844320aa2ba2a457 P cb587703ffb2471cb304f6d624d407b9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:05.769724 20871 raft_consensus.cc:2454] T d5e40dacbb7c4670844320aa2ba2a457 P bf39e1fb29a747089147b7cb4af631c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0c828730d04c65a1bca1543e765658 in term 1.
I20260810 07:15:05.769716 21154 raft_consensus.cc:2454] T d5e40dacbb7c4670844320aa2ba2a457 P cb587703ffb2471cb304f6d624d407b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0c828730d04c65a1bca1543e765658 in term 1.
I20260810 07:15:05.769901 20944 leader_election.cc:302] T d5e40dacbb7c4670844320aa2ba2a457 P cb0c828730d04c65a1bca1543e765658 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb0c828730d04c65a1bca1543e765658, cb587703ffb2471cb304f6d624d407b9; no voters:
I20260810 07:15:05.770000 21231 raft_consensus.cc:2781] T d5e40dacbb7c4670844320aa2ba2a457 P cb0c828730d04c65a1bca1543e765658 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:05.770150 21231 raft_consensus.cc:686] T d5e40dacbb7c4670844320aa2ba2a457 P cb0c828730d04c65a1bca1543e765658 [term 1 LEADER]: Becoming Leader. State: Replica: cb0c828730d04c65a1bca1543e765658, State: Running, Role: LEADER
I20260810 07:15:05.770267 21231 consensus_queue.cc:237] T d5e40dacbb7c4670844320aa2ba2a457 P cb0c828730d04c65a1bca1543e765658 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb0c828730d04c65a1bca1543e765658" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44807 } } peers { permanent_uuid: "cb587703ffb2471cb304f6d624d407b9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 46285 } } peers { permanent_uuid: "bf39e1fb29a747089147b7cb4af631c2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 46347 } }
I20260810 07:15:05.771184 20718 catalog_manager.cc:5495] T d5e40dacbb7c4670844320aa2ba2a457 P cb0c828730d04c65a1bca1543e765658 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb0c828730d04c65a1bca1543e765658 (127.15.217.66). New cstate: current_term: 1 leader_uuid: "cb0c828730d04c65a1bca1543e765658" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb0c828730d04c65a1bca1543e765658" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb587703ffb2471cb304f6d624d407b9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 46285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf39e1fb29a747089147b7cb4af631c2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 46347 } health_report { overall_health: UNKNOWN } } }
I20260810 07:15:05.863615 21240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:05.860881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58642 (local address 127.15.217.66:44807)
0810 07:15:05.861005 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:05.861008 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:05.861058 (+ 50us) server_negotiation.cc:407] Connection header received
0810 07:15:05.861150 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:05.861151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:05.861175 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:05.861269 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:15:05.861427 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.861942 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.862747 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.862970 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.863047 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:05.863235 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:15:05.863373 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:15:05.863475 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:05.863493 (+ 18us) server_negotiation.cc:299] Negotiation successful
0810 07:15:05.863516 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":62,"threads_started":1}
I20260810 07:15:05.865518 20871 raft_consensus.cc:1261] T d5e40dacbb7c4670844320aa2ba2a457 P bf39e1fb29a747089147b7cb4af631c2 [term 1 FOLLOWER]: Refusing update from remote peer cb0c828730d04c65a1bca1543e765658: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:05.865546 21154 raft_consensus.cc:1261] T d5e40dacbb7c4670844320aa2ba2a457 P cb587703ffb2471cb304f6d624d407b9 [term 1 FOLLOWER]: Refusing update from remote peer cb0c828730d04c65a1bca1543e765658: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:05.865947 21236 consensus_queue.cc:1035] T d5e40dacbb7c4670844320aa2ba2a457 P cb0c828730d04c65a1bca1543e765658 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf39e1fb29a747089147b7cb4af631c2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 46347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:05.866101 21231 consensus_queue.cc:1035] T d5e40dacbb7c4670844320aa2ba2a457 P cb0c828730d04c65a1bca1543e765658 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb587703ffb2471cb304f6d624d407b9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 46285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:05.869678 21244 mvcc.cc:205] Tried to move back new op lower bound from 7316873649622212608 to 7316873649235767296. Current Snapshot: MvccSnapshot[applied={T|T < 7316873649622212608 or (T in {7316873649622212608})}]
I20260810 07:15:05.871737 20718 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:05.871837 20718 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:05.872653 20718 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 170460c06ef54054bcaf901baaa06408: Sending DeleteTablet for 3 replicas of tablet d5e40dacbb7c4670844320aa2ba2a457
I20260810 07:15:05.872960 20990 tablet_service.cc:1514] Processing DeleteTablet for tablet d5e40dacbb7c4670844320aa2ba2a457 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:05 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:58638
I20260810 07:15:05.873044 21133 tablet_service.cc:1514] Processing DeleteTablet for tablet d5e40dacbb7c4670844320aa2ba2a457 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:05 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:48512
I20260810 07:15:05.873196 20851 tablet_service.cc:1514] Processing DeleteTablet for tablet d5e40dacbb7c4670844320aa2ba2a457 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:05 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:44354
I20260810 07:15:05.873309 21256 tablet_replica.cc:330] T d5e40dacbb7c4670844320aa2ba2a457 P cb587703ffb2471cb304f6d624d407b9: stopping tablet replica
I20260810 07:15:05.873411 21256 raft_consensus.cc:2229] T d5e40dacbb7c4670844320aa2ba2a457 P cb587703ffb2471cb304f6d624d407b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:05.873505 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 20784
I20260810 07:15:05.874940 21256 raft_consensus.cc:2258] T d5e40dacbb7c4670844320aa2ba2a457 P cb587703ffb2471cb304f6d624d407b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:05.875633 21256 ts_tablet_manager.cc:1905] T d5e40dacbb7c4670844320aa2ba2a457 P cb587703ffb2471cb304f6d624d407b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:05.877332 21256 ts_tablet_manager.cc:1918] T d5e40dacbb7c4670844320aa2ba2a457 P cb587703ffb2471cb304f6d624d407b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:15:05.877424 21256 log.cc:1196] T d5e40dacbb7c4670844320aa2ba2a457 P cb587703ffb2471cb304f6d624d407b9: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-2/wal/wals/d5e40dacbb7c4670844320aa2ba2a457
I20260810 07:15:05.877661 21256 ts_tablet_manager.cc:1939] T d5e40dacbb7c4670844320aa2ba2a457 P cb587703ffb2471cb304f6d624d407b9: Deleting consensus metadata
I20260810 07:15:05.878105 20705 catalog_manager.cc:4841] TS cb587703ffb2471cb304f6d624d407b9 (127.15.217.67:46285): tablet d5e40dacbb7c4670844320aa2ba2a457 (table test-table [id=9d6008c7effd4fd385c8417785c0707c]) successfully deleted
I20260810 07:15:05.880692 21254 tablet_replica.cc:330] T d5e40dacbb7c4670844320aa2ba2a457 P cb0c828730d04c65a1bca1543e765658: stopping tablet replica
I20260810 07:15:05.880836 21254 raft_consensus.cc:2229] T d5e40dacbb7c4670844320aa2ba2a457 P cb0c828730d04c65a1bca1543e765658 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:15:05.881047 21254 raft_consensus.cc:2258] T d5e40dacbb7c4670844320aa2ba2a457 P cb0c828730d04c65a1bca1543e765658 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:05.881412 21254 ts_tablet_manager.cc:1905] T d5e40dacbb7c4670844320aa2ba2a457 P cb0c828730d04c65a1bca1543e765658: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:05.882977 21254 ts_tablet_manager.cc:1918] T d5e40dacbb7c4670844320aa2ba2a457 P cb0c828730d04c65a1bca1543e765658: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:15:05.883073 21254 log.cc:1196] T d5e40dacbb7c4670844320aa2ba2a457 P cb0c828730d04c65a1bca1543e765658: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346081759552-16229-0/minicluster-data/ts-1/wal/wals/d5e40dacbb7c4670844320aa2ba2a457
W20260810 07:15:05.883200 20705 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.217.65:46347 (error 108)
I20260810 07:15:05.883504 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 20923
I20260810 07:15:05.883584 21254 ts_tablet_manager.cc:1939] T d5e40dacbb7c4670844320aa2ba2a457 P cb0c828730d04c65a1bca1543e765658: Deleting consensus metadata
I20260810 07:15:05.888737 21260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:05.883587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:46347 (local address 127.0.0.1:44368)
0810 07:15:05.888630 (+ 5043us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:05.888693 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.65:46347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5021,"thread_start_us":46,"threads_started":1}
W20260810 07:15:05.888895 20705 catalog_manager.cc:4584] TS bf39e1fb29a747089147b7cb4af631c2 (127.15.217.65:46347): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d5e40dacbb7c4670844320aa2ba2a457: Network error: Client connection negotiation failed: client connection to 127.15.217.65:46347: connect: Connection refused (error 111)
I20260810 07:15:05.890990 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 21065
I20260810 07:15:05.891077 21260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:05.890957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:44807 (local address 127.0.0.1:58650)
0810 07:15:05.891018 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:05.891050 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.66:44807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 07:15:05.898162 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 20683
2026-08-10T07:15:05Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2540 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-10T07:15:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:15:05Z Disabled control of system clock
I20260810 07:15:05.930529 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:43801
--webserver_interface=127.15.217.126
--webserver_port=0
--builtin_ntp_servers=127.15.217.84:43445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:43801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 07:15:06.015612 21264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:06.015841 21264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:06.015863 21264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:06.016999 21264 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:06.017040 21264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:06.017053 21264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:06.017066 21264 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:06.017077 21264 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:15:06.018417 21264 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:43445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:43801
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:43801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:06.018651 21264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:06.018914 21264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:06.024250 21269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:06.024257 21271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:06.024363 21264 server_base.cc:1034] running on GCE node
W20260810 07:15:06.024314 21273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:06.024788 21264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:06.025134 21264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:06.026321 21264 hybrid_clock.cc:648] HybridClock initialized: now 1786346106026312 us; error 38 us; skew 500 ppm
I20260810 07:15:06.027685 21264 webserver.cc:460] Webserver started at http://127.15.217.126:42785/ using document root <none> and password file <none>
I20260810 07:15:06.027969 21264 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:06.028064 21264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:06.028236 21264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:06.029517 21264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "7fe96d3f035a4fd9af2505e033283fdb"
format_stamp: "Formatted at 2026-08-10 07:15:06 on dist-test-slave-10pc"
server_key: "e91c142f9b15d567a6c6c5d2afd3d260"
server_key_iv: "e3c512560790b103b5de2ee90fa93e3d"
server_key_version: "encryptionkey@0"
I20260810 07:15:06.029948 21264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "7fe96d3f035a4fd9af2505e033283fdb"
format_stamp: "Formatted at 2026-08-10 07:15:06 on dist-test-slave-10pc"
server_key: "e91c142f9b15d567a6c6c5d2afd3d260"
server_key_iv: "e3c512560790b103b5de2ee90fa93e3d"
server_key_version: "encryptionkey@0"
I20260810 07:15:06.031395 21264 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:15:06.032153 21280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:06.032334 21264 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 07:15:06.032490 21264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "7fe96d3f035a4fd9af2505e033283fdb"
format_stamp: "Formatted at 2026-08-10 07:15:06 on dist-test-slave-10pc"
server_key: "e91c142f9b15d567a6c6c5d2afd3d260"
server_key_iv: "e3c512560790b103b5de2ee90fa93e3d"
server_key_version: "encryptionkey@0"
I20260810 07:15:06.032588 21264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:06.036289 21264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:06.036659 21264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:06.036830 21264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:06.041191 21264 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:43801
I20260810 07:15:06.041222 21334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:43801 every 8 connection(s)
I20260810 07:15:06.041603 21264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:15:06.042294 21335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:06.045243 21335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fe96d3f035a4fd9af2505e033283fdb: Bootstrap starting.
I20260810 07:15:06.045341 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 21264
I20260810 07:15:06.045480 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:15:06.045676 16229 external_mini_cluster.cc:1428] Setting key c3363e05b13fff4d8ceceff885f9f84a
I20260810 07:15:06.046001 21335 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7fe96d3f035a4fd9af2505e033283fdb: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:06.046401 21335 log.cc:824] T 00000000000000000000000000000000 P 7fe96d3f035a4fd9af2505e033283fdb: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:06.047228 21335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fe96d3f035a4fd9af2505e033283fdb: No bootstrap required, opened a new log
I20260810 07:15:06.048617 21339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:06.046274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50466 (local address 127.15.217.126:43801)
0810 07:15:06.046500 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:06.046508 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:06.046543 (+ 35us) server_negotiation.cc:407] Connection header received
0810 07:15:06.047118 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:06.047128 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:06.047179 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:06.047341 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:06.048120 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:06.048124 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:06.048124 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:06.048129 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:06.048177 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:06.048191 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:06.048294 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:06.048325 (+ 31us) server_negotiation.cc:299] Negotiation successful
0810 07:15:06.048385 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":90,"threads_started":1}
I20260810 07:15:06.049177 21335 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7fe96d3f035a4fd9af2505e033283fdb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fe96d3f035a4fd9af2505e033283fdb" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 43801 } }
I20260810 07:15:06.049348 21335 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7fe96d3f035a4fd9af2505e033283fdb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:06.049388 21335 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7fe96d3f035a4fd9af2505e033283fdb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fe96d3f035a4fd9af2505e033283fdb, State: Initialized, Role: FOLLOWER
I20260810 07:15:06.049511 21335 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7fe96d3f035a4fd9af2505e033283fdb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fe96d3f035a4fd9af2505e033283fdb" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 43801 } }
I20260810 07:15:06.049594 21335 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7fe96d3f035a4fd9af2505e033283fdb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:15:06.049616 21335 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7fe96d3f035a4fd9af2505e033283fdb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:15:06.049682 21335 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7fe96d3f035a4fd9af2505e033283fdb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:06.050549 21335 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7fe96d3f035a4fd9af2505e033283fdb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fe96d3f035a4fd9af2505e033283fdb" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 43801 } }
I20260810 07:15:06.050642 21335 leader_election.cc:302] T 00000000000000000000000000000000 P 7fe96d3f035a4fd9af2505e033283fdb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7fe96d3f035a4fd9af2505e033283fdb; no voters:
I20260810 07:15:06.050817 21335 leader_election.cc:288] T 00000000000000000000000000000000 P 7fe96d3f035a4fd9af2505e033283fdb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:15:06.050899 21341 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7fe96d3f035a4fd9af2505e033283fdb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:06.051070 21341 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7fe96d3f035a4fd9af2505e033283fdb [term 1 LEADER]: Becoming Leader. State: Replica: 7fe96d3f035a4fd9af2505e033283fdb, State: Running, Role: LEADER
I20260810 07:15:06.051091 21335 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7fe96d3f035a4fd9af2505e033283fdb [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:06.051186 21341 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7fe96d3f035a4fd9af2505e033283fdb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fe96d3f035a4fd9af2505e033283fdb" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 43801 } }
I20260810 07:15:06.051615 21344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fe96d3f035a4fd9af2505e033283fdb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7fe96d3f035a4fd9af2505e033283fdb. Latest consensus state: current_term: 1 leader_uuid: "7fe96d3f035a4fd9af2505e033283fdb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fe96d3f035a4fd9af2505e033283fdb" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 43801 } } }
I20260810 07:15:06.051607 21342 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fe96d3f035a4fd9af2505e033283fdb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7fe96d3f035a4fd9af2505e033283fdb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fe96d3f035a4fd9af2505e033283fdb" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 43801 } } }
I20260810 07:15:06.051731 21344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fe96d3f035a4fd9af2505e033283fdb [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:06.051740 21342 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fe96d3f035a4fd9af2505e033283fdb [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:06.052016 21352 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:06.052634 21352 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:15:06.054220 21352 catalog_manager.cc:1347] Generated new cluster ID: d5a3a844e9e1470da2bdce964212ed18
I20260810 07:15:06.054279 21352 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:15:06.069517 21352 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:15:06.069746 21352 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:06.075884 21352 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7fe96d3f035a4fd9af2505e033283fdb: Generated new TSK 0
I20260810 07:15:06.076114 21352 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:15:06.079586 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.65:0
--local_ip_for_outbound_sockets=127.15.217.65
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:43801
--builtin_ntp_servers=127.15.217.84:43445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 07:15:06.178016 21362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:06.178248 21362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:06.178277 21362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:06.179615 21362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:06.179662 21362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:06.179693 21362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.65
I20260810 07:15:06.181329 21362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:43445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:43801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.65
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:06.181623 21362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:06.181977 21362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:06.187563 21367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:06.187563 21368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:06.187731 21370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:06.187809 21362 server_base.cc:1034] running on GCE node
I20260810 07:15:06.188030 21362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:06.188293 21362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:06.189463 21362 hybrid_clock.cc:648] HybridClock initialized: now 1786346106189434 us; error 56 us; skew 500 ppm
I20260810 07:15:06.190806 21362 webserver.cc:460] Webserver started at http://127.15.217.65:39057/ using document root <none> and password file <none>
I20260810 07:15:06.191058 21362 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:06.191133 21362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:06.191289 21362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:06.192433 21362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-0/data/instance:
uuid: "47327cfe57474293a65b1f5d1016a724"
format_stamp: "Formatted at 2026-08-10 07:15:06 on dist-test-slave-10pc"
server_key: "5cea91a605e419722980960e76e0f410"
server_key_iv: "d2365e569e0470a294550a6793b82f7f"
server_key_version: "encryptionkey@0"
I20260810 07:15:06.192888 21362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance:
uuid: "47327cfe57474293a65b1f5d1016a724"
format_stamp: "Formatted at 2026-08-10 07:15:06 on dist-test-slave-10pc"
server_key: "5cea91a605e419722980960e76e0f410"
server_key_iv: "d2365e569e0470a294550a6793b82f7f"
server_key_version: "encryptionkey@0"
I20260810 07:15:06.194607 21362 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:06.195439 21377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:06.195664 21362 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:06.195770 21362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-0/wal
uuid: "47327cfe57474293a65b1f5d1016a724"
format_stamp: "Formatted at 2026-08-10 07:15:06 on dist-test-slave-10pc"
server_key: "5cea91a605e419722980960e76e0f410"
server_key_iv: "d2365e569e0470a294550a6793b82f7f"
server_key_version: "encryptionkey@0"
I20260810 07:15:06.195860 21362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:06.199319 21362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:06.199612 21362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:06.199766 21362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:06.199982 21362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:06.200326 21362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:06.200382 21362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:06.200423 21362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:06.200472 21362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:06.206420 21362 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.65:44957
I20260810 07:15:06.206511 21501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.65:44957 every 8 connection(s)
I20260810 07:15:06.206878 21362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
I20260810 07:15:06.208112 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 21362
I20260810 07:15:06.208225 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance
I20260810 07:15:06.208433 16229 external_mini_cluster.cc:1428] Setting key 76c0bb8c2fce335803aabc245ccade3a
I20260810 07:15:06.209503 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.66:0
--local_ip_for_outbound_sockets=127.15.217.66
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:43801
--builtin_ntp_servers=127.15.217.84:43445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260810 07:15:06.210455 21339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:06.207697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:33571 (local address 127.15.217.126:43801)
0810 07:15:06.207787 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:06.207789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:06.208043 (+ 254us) server_negotiation.cc:407] Connection header received
0810 07:15:06.208701 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:06.208703 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:06.208710 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:06.208754 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:06.210209 (+ 1455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:06.210210 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:06.210210 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:06.210216 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:06.210232 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:06.210237 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:06.210291 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:06.210367 (+ 76us) server_negotiation.cc:299] Negotiation successful
0810 07:15:06.210382 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 07:15:06.210666 21504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:06.207667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43801 (local address 127.15.217.65:33571)
0810 07:15:06.207897 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:06.207946 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:06.208601 (+ 655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:06.208839 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:06.208843 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:06.208853 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:06.210036 (+ 1183us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:15:06.210037 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:06.210068 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:15:06.210098 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:06.210296 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:06.210301 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:06.210305 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:15:06.210381 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 07:15:06.210447 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":127,"threads_started":1}
I20260810 07:15:06.211414 21502 heartbeater.cc:347] Connected to a master server at 127.15.217.126:43801
I20260810 07:15:06.211527 21502 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:06.211705 21502 heartbeater.cc:511] Master 127.15.217.126:43801 requested a full tablet report, sending...
I20260810 07:15:06.212234 21297 ts_manager.cc:194] Registered new tserver with Master: 47327cfe57474293a65b1f5d1016a724 (127.15.217.65:44957)
W20260810 07:15:06.299949 21505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:06.300174 21505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:06.300199 21505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:06.301426 21505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:06.301469 21505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:06.301496 21505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.66
I20260810 07:15:06.302897 21505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:43445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:43801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.66
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:06.303169 21505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:06.303488 21505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:06.308771 21513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:06.308811 21510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:06.308781 21511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:06.308935 21505 server_base.cc:1034] running on GCE node
I20260810 07:15:06.309139 21505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:06.309357 21505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:06.310505 21505 hybrid_clock.cc:648] HybridClock initialized: now 1786346106310467 us; error 54 us; skew 500 ppm
I20260810 07:15:06.311770 21505 webserver.cc:460] Webserver started at http://127.15.217.66:45889/ using document root <none> and password file <none>
I20260810 07:15:06.312023 21505 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:06.312071 21505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:06.312252 21505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:06.313463 21505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-1/data/instance:
uuid: "8862807f605e42c9b2fb5277d15695a1"
format_stamp: "Formatted at 2026-08-10 07:15:06 on dist-test-slave-10pc"
server_key: "320dab3919790ba37333ac051e6c2ce0"
server_key_iv: "0c37f566ebdc2f9eaf00c933959c5dc9"
server_key_version: "encryptionkey@0"
I20260810 07:15:06.313864 21505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance:
uuid: "8862807f605e42c9b2fb5277d15695a1"
format_stamp: "Formatted at 2026-08-10 07:15:06 on dist-test-slave-10pc"
server_key: "320dab3919790ba37333ac051e6c2ce0"
server_key_iv: "0c37f566ebdc2f9eaf00c933959c5dc9"
server_key_version: "encryptionkey@0"
I20260810 07:15:06.315333 21505 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:15:06.316195 21519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:06.316416 21505 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:06.316556 21505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-1/wal
uuid: "8862807f605e42c9b2fb5277d15695a1"
format_stamp: "Formatted at 2026-08-10 07:15:06 on dist-test-slave-10pc"
server_key: "320dab3919790ba37333ac051e6c2ce0"
server_key_iv: "0c37f566ebdc2f9eaf00c933959c5dc9"
server_key_version: "encryptionkey@0"
I20260810 07:15:06.316637 21505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:06.320010 21505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:06.320309 21505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:06.320456 21505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:06.320729 21505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:06.321069 21505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:06.321102 21505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:06.321149 21505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:06.321177 21505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:06.327376 21505 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.66:33497
I20260810 07:15:06.327459 21637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.66:33497 every 8 connection(s)
I20260810 07:15:06.327841 21505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
I20260810 07:15:06.331019 21339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:06.328756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:33923 (local address 127.15.217.126:43801)
0810 07:15:06.328831 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:06.328834 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:06.329073 (+ 239us) server_negotiation.cc:407] Connection header received
0810 07:15:06.329565 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:06.329566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:06.329572 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:06.329636 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:06.330795 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:06.330797 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:06.330797 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:06.330801 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:06.330818 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:06.330822 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:06.330871 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:06.330947 (+ 76us) server_negotiation.cc:299] Negotiation successful
0810 07:15:06.330960 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 07:15:06.331214 21640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:06.328741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43801 (local address 127.15.217.66:33923)
0810 07:15:06.328935 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:06.328979 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:06.329479 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:06.329701 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:06.329705 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:06.329714 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:06.330637 (+ 923us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:15:06.330638 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:06.330670 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:15:06.330702 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:06.330870 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:06.330872 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:06.330875 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:15:06.330928 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 07:15:06.330979 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":87,"threads_started":1}
I20260810 07:15:06.331744 21638 heartbeater.cc:347] Connected to a master server at 127.15.217.126:43801
I20260810 07:15:06.331873 21638 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:06.332041 21638 heartbeater.cc:511] Master 127.15.217.126:43801 requested a full tablet report, sending...
I20260810 07:15:06.332525 21297 ts_manager.cc:194] Registered new tserver with Master: 8862807f605e42c9b2fb5277d15695a1 (127.15.217.66:33497)
I20260810 07:15:06.336267 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 21505
I20260810 07:15:06.336345 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance
I20260810 07:15:06.336555 16229 external_mini_cluster.cc:1428] Setting key 18278113335321895919862f344606ca
I20260810 07:15:06.337500 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.67:0
--local_ip_for_outbound_sockets=127.15.217.67
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:43801
--builtin_ntp_servers=127.15.217.84:43445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 07:15:06.424387 21641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:06.424695 21641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:06.424733 21641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:06.425936 21641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:06.426002 21641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:06.426043 21641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.67
I20260810 07:15:06.427412 21641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:43445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:43801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.67
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:06.427680 21641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:06.427989 21641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:06.433063 21647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:06.433240 21641 server_base.cc:1034] running on GCE node
W20260810 07:15:06.433071 21648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:06.433071 21650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:06.433660 21641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:06.433912 21641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:06.435096 21641 hybrid_clock.cc:648] HybridClock initialized: now 1786346106435084 us; error 72 us; skew 500 ppm
I20260810 07:15:06.436609 21641 webserver.cc:460] Webserver started at http://127.15.217.67:34061/ using document root <none> and password file <none>
I20260810 07:15:06.436869 21641 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:06.436937 21641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:06.437083 21641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:06.438237 21641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-2/data/instance:
uuid: "34610ddda6f7449a9cd55b3310457b3e"
format_stamp: "Formatted at 2026-08-10 07:15:06 on dist-test-slave-10pc"
server_key: "5d6cb1ea64b7d59f16bcb13d1e545102"
server_key_iv: "87d67056c8e53177db86de15aa7e75bf"
server_key_version: "encryptionkey@0"
I20260810 07:15:06.438632 21641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance:
uuid: "34610ddda6f7449a9cd55b3310457b3e"
format_stamp: "Formatted at 2026-08-10 07:15:06 on dist-test-slave-10pc"
server_key: "5d6cb1ea64b7d59f16bcb13d1e545102"
server_key_iv: "87d67056c8e53177db86de15aa7e75bf"
server_key_version: "encryptionkey@0"
I20260810 07:15:06.440002 21641 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:15:06.440840 21659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:06.441022 21641 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:06.441134 21641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-2/wal
uuid: "34610ddda6f7449a9cd55b3310457b3e"
format_stamp: "Formatted at 2026-08-10 07:15:06 on dist-test-slave-10pc"
server_key: "5d6cb1ea64b7d59f16bcb13d1e545102"
server_key_iv: "87d67056c8e53177db86de15aa7e75bf"
server_key_version: "encryptionkey@0"
I20260810 07:15:06.441217 21641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:06.444653 21641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:06.444952 21641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:06.445094 21641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:06.445312 21641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:06.445633 21641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:06.445690 21641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:06.445730 21641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:06.445757 21641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:06.451812 21641 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.67:38067
I20260810 07:15:06.451897 21778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.67:38067 every 8 connection(s)
I20260810 07:15:06.452252 21641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
I20260810 07:15:06.453671 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 21641
I20260810 07:15:06.453775 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance
I20260810 07:15:06.453980 16229 external_mini_cluster.cc:1428] Setting key 77469bc04e9dffb53c969b17347e7b28
I20260810 07:15:06.455927 21339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:06.453148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:32937 (local address 127.15.217.126:43801)
0810 07:15:06.453220 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:06.453223 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:06.453530 (+ 307us) server_negotiation.cc:407] Connection header received
0810 07:15:06.454349 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:06.454351 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:06.454373 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:06.454456 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:06.455722 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:06.455723 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:06.455724 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:06.455728 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:06.455746 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:06.455750 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:06.455807 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:06.455858 (+ 51us) server_negotiation.cc:299] Negotiation successful
0810 07:15:06.455870 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 07:15:06.456153 21781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:06.453163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43801 (local address 127.15.217.67:32937)
0810 07:15:06.453361 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:06.453395 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:06.454202 (+ 807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:06.454539 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:06.454543 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:06.454554 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:06.455574 (+ 1020us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:15:06.455575 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:06.455605 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:15:06.455634 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:06.455808 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:06.455810 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:06.455813 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:15:06.455863 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 07:15:06.455919 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":90,"threads_started":1}
I20260810 07:15:06.456704 21779 heartbeater.cc:347] Connected to a master server at 127.15.217.126:43801
I20260810 07:15:06.456789 21779 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:06.456964 21779 heartbeater.cc:511] Master 127.15.217.126:43801 requested a full tablet report, sending...
I20260810 07:15:06.457371 21297 ts_manager.cc:194] Registered new tserver with Master: 34610ddda6f7449a9cd55b3310457b3e (127.15.217.67:38067)
I20260810 07:15:06.466318 16229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:15:06.469254 21339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:06.467903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53424 (local address 127.15.217.126:43801)
0810 07:15:06.467990 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:06.467991 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:06.468156 (+ 165us) server_negotiation.cc:407] Connection header received
0810 07:15:06.468212 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:06.468213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:06.468219 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:06.468281 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:06.469019 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:06.469020 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:06.469021 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:06.469025 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:06.469039 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:06.469043 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:06.469134 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:06.469180 (+ 46us) server_negotiation.cc:299] Negotiation successful
0810 07:15:06.469192 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 07:15:06.471276 21297 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53424:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:15:06.473078 21297 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:15:06.477595 21790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:06.475786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33134 (local address 127.15.217.66:33497)
0810 07:15:06.476055 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:06.476059 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:06.476345 (+ 286us) server_negotiation.cc:407] Connection header received
0810 07:15:06.476377 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:06.476379 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:06.476388 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:06.476490 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:06.477253 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:06.477262 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:06.477262 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:06.477266 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:06.477308 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:06.477328 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:06.477401 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:06.477487 (+ 86us) server_negotiation.cc:299] Negotiation successful
0810 07:15:06.477503 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":79,"threads_started":1}
I20260810 07:15:06.477589 21791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:06.475571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:33497 (local address 127.0.0.1:33134)
0810 07:15:06.476238 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:06.476248 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:06.476290 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:06.476760 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:06.476761 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:06.476763 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:06.477091 (+ 328us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:15:06.477092 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:06.477127 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:15:06.477156 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:06.477394 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:06.477402 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:06.477406 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:15:06.477469 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 07:15:06.477485 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"mutex_wait_us":2,"thread_start_us":427,"threads_started":1}
I20260810 07:15:06.477753 21789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:06.475769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49086 (local address 127.15.217.65:44957)
0810 07:15:06.475966 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:06.475972 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:06.476142 (+ 170us) server_negotiation.cc:407] Connection header received
0810 07:15:06.476216 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:06.476220 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:06.476232 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:06.476311 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:06.477261 (+ 950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:06.477271 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:06.477271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:06.477276 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:06.477330 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:06.477360 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:06.477435 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:06.477634 (+ 199us) server_negotiation.cc:299] Negotiation successful
0810 07:15:06.477661 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":101,"threads_started":1}
I20260810 07:15:06.477885 21792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:06.475571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:44957 (local address 127.0.0.1:49086)
0810 07:15:06.476072 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:06.476083 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:06.476139 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:06.476711 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:06.476712 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:06.476716 (+ 4us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:06.477145 (+ 429us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:15:06.477145 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:06.477149 (+ 4us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:15:06.477156 (+ 7us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:06.477467 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:06.477468 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:06.477469 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 07:15:06.477516 (+ 47us) client_negotiation.cc:240] Negotiation successful
0810 07:15:06.477531 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"spinlock_wait_cycles":896,"thread_start_us":432,"threads_started":1}
I20260810 07:15:06.478345 21788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:06.475659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59248 (local address 127.15.217.67:38067)
0810 07:15:06.476113 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:06.476116 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:06.476126 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:15:06.476151 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:06.476152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:06.476161 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:06.476254 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:06.477713 (+ 1459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:06.477721 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:06.477721 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:06.477725 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:06.477758 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:06.477782 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:06.478211 (+ 429us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:06.478242 (+ 31us) server_negotiation.cc:299] Negotiation successful
0810 07:15:06.478261 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":113,"threads_started":1}
I20260810 07:15:06.478067 21787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:06.475568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:38067 (local address 127.0.0.1:59248)
0810 07:15:06.475823 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:06.475869 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:06.475931 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:06.476491 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:06.476500 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:06.476508 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:06.477609 (+ 1101us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:15:06.477609 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:06.477616 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:15:06.477629 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:06.477897 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:06.477898 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:06.477899 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 07:15:06.477977 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 07:15:06.477992 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":70,"threads_started":1}
I20260810 07:15:06.478819 21570 tablet_service.cc:1467] Processing CreateTablet for tablet 7aaebde43b554d51aff8c8489e6178d3 (DEFAULT_TABLE table=test-table [id=feec631940b34a259b83ee0d02a6e4d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:06.479250 21570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7aaebde43b554d51aff8c8489e6178d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:06.479200 21709 tablet_service.cc:1467] Processing CreateTablet for tablet 7aaebde43b554d51aff8c8489e6178d3 (DEFAULT_TABLE table=test-table [id=feec631940b34a259b83ee0d02a6e4d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:06.479566 21709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7aaebde43b554d51aff8c8489e6178d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:06.480981 21430 tablet_service.cc:1467] Processing CreateTablet for tablet 7aaebde43b554d51aff8c8489e6178d3 (DEFAULT_TABLE table=test-table [id=feec631940b34a259b83ee0d02a6e4d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:06.481385 21430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7aaebde43b554d51aff8c8489e6178d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:06.483168 21793 tablet_bootstrap.cc:492] T 7aaebde43b554d51aff8c8489e6178d3 P 34610ddda6f7449a9cd55b3310457b3e: Bootstrap starting.
I20260810 07:15:06.483807 21793 tablet_bootstrap.cc:654] T 7aaebde43b554d51aff8c8489e6178d3 P 34610ddda6f7449a9cd55b3310457b3e: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:06.484114 21793 log.cc:824] T 7aaebde43b554d51aff8c8489e6178d3 P 34610ddda6f7449a9cd55b3310457b3e: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:06.485016 21793 tablet_bootstrap.cc:492] T 7aaebde43b554d51aff8c8489e6178d3 P 34610ddda6f7449a9cd55b3310457b3e: No bootstrap required, opened a new log
I20260810 07:15:06.485096 21793 ts_tablet_manager.cc:1397] T 7aaebde43b554d51aff8c8489e6178d3 P 34610ddda6f7449a9cd55b3310457b3e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:15:06.487018 21797 tablet_bootstrap.cc:492] T 7aaebde43b554d51aff8c8489e6178d3 P 8862807f605e42c9b2fb5277d15695a1: Bootstrap starting.
I20260810 07:15:06.487604 21797 tablet_bootstrap.cc:654] T 7aaebde43b554d51aff8c8489e6178d3 P 8862807f605e42c9b2fb5277d15695a1: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:06.487738 21793 raft_consensus.cc:348] T 7aaebde43b554d51aff8c8489e6178d3 P 34610ddda6f7449a9cd55b3310457b3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47327cfe57474293a65b1f5d1016a724" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 44957 } } peers { permanent_uuid: "34610ddda6f7449a9cd55b3310457b3e" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 38067 } } peers { permanent_uuid: "8862807f605e42c9b2fb5277d15695a1" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33497 } }
I20260810 07:15:06.487891 21793 raft_consensus.cc:374] T 7aaebde43b554d51aff8c8489e6178d3 P 34610ddda6f7449a9cd55b3310457b3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:06.487917 21793 raft_consensus.cc:729] T 7aaebde43b554d51aff8c8489e6178d3 P 34610ddda6f7449a9cd55b3310457b3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34610ddda6f7449a9cd55b3310457b3e, State: Initialized, Role: FOLLOWER
I20260810 07:15:06.487953 21797 log.cc:824] T 7aaebde43b554d51aff8c8489e6178d3 P 8862807f605e42c9b2fb5277d15695a1: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:06.488075 21793 consensus_queue.cc:260] T 7aaebde43b554d51aff8c8489e6178d3 P 34610ddda6f7449a9cd55b3310457b3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47327cfe57474293a65b1f5d1016a724" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 44957 } } peers { permanent_uuid: "34610ddda6f7449a9cd55b3310457b3e" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 38067 } } peers { permanent_uuid: "8862807f605e42c9b2fb5277d15695a1" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33497 } }
I20260810 07:15:06.488392 21793 ts_tablet_manager.cc:1428] T 7aaebde43b554d51aff8c8489e6178d3 P 34610ddda6f7449a9cd55b3310457b3e: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260810 07:15:06.488471 21779 heartbeater.cc:503] Master 127.15.217.126:43801 was elected leader, sending a full tablet report...
I20260810 07:15:06.488533 21796 tablet_bootstrap.cc:492] T 7aaebde43b554d51aff8c8489e6178d3 P 47327cfe57474293a65b1f5d1016a724: Bootstrap starting.
I20260810 07:15:06.488839 21797 tablet_bootstrap.cc:492] T 7aaebde43b554d51aff8c8489e6178d3 P 8862807f605e42c9b2fb5277d15695a1: No bootstrap required, opened a new log
I20260810 07:15:06.488915 21797 ts_tablet_manager.cc:1397] T 7aaebde43b554d51aff8c8489e6178d3 P 8862807f605e42c9b2fb5277d15695a1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:15:06.489315 21796 tablet_bootstrap.cc:654] T 7aaebde43b554d51aff8c8489e6178d3 P 47327cfe57474293a65b1f5d1016a724: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:06.489653 21796 log.cc:824] T 7aaebde43b554d51aff8c8489e6178d3 P 47327cfe57474293a65b1f5d1016a724: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:06.490574 21796 tablet_bootstrap.cc:492] T 7aaebde43b554d51aff8c8489e6178d3 P 47327cfe57474293a65b1f5d1016a724: No bootstrap required, opened a new log
I20260810 07:15:06.490630 21796 ts_tablet_manager.cc:1397] T 7aaebde43b554d51aff8c8489e6178d3 P 47327cfe57474293a65b1f5d1016a724: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260810 07:15:06.491389 21797 raft_consensus.cc:348] T 7aaebde43b554d51aff8c8489e6178d3 P 8862807f605e42c9b2fb5277d15695a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47327cfe57474293a65b1f5d1016a724" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 44957 } } peers { permanent_uuid: "34610ddda6f7449a9cd55b3310457b3e" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 38067 } } peers { permanent_uuid: "8862807f605e42c9b2fb5277d15695a1" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33497 } }
I20260810 07:15:06.491549 21797 raft_consensus.cc:374] T 7aaebde43b554d51aff8c8489e6178d3 P 8862807f605e42c9b2fb5277d15695a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:06.491595 21797 raft_consensus.cc:729] T 7aaebde43b554d51aff8c8489e6178d3 P 8862807f605e42c9b2fb5277d15695a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8862807f605e42c9b2fb5277d15695a1, State: Initialized, Role: FOLLOWER
I20260810 07:15:06.491726 21797 consensus_queue.cc:260] T 7aaebde43b554d51aff8c8489e6178d3 P 8862807f605e42c9b2fb5277d15695a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47327cfe57474293a65b1f5d1016a724" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 44957 } } peers { permanent_uuid: "34610ddda6f7449a9cd55b3310457b3e" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 38067 } } peers { permanent_uuid: "8862807f605e42c9b2fb5277d15695a1" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33497 } }
I20260810 07:15:06.492014 21797 ts_tablet_manager.cc:1428] T 7aaebde43b554d51aff8c8489e6178d3 P 8862807f605e42c9b2fb5277d15695a1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:15:06.492046 21638 heartbeater.cc:503] Master 127.15.217.126:43801 was elected leader, sending a full tablet report...
I20260810 07:15:06.493008 21796 raft_consensus.cc:348] T 7aaebde43b554d51aff8c8489e6178d3 P 47327cfe57474293a65b1f5d1016a724 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47327cfe57474293a65b1f5d1016a724" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 44957 } } peers { permanent_uuid: "34610ddda6f7449a9cd55b3310457b3e" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 38067 } } peers { permanent_uuid: "8862807f605e42c9b2fb5277d15695a1" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33497 } }
I20260810 07:15:06.493194 21796 raft_consensus.cc:374] T 7aaebde43b554d51aff8c8489e6178d3 P 47327cfe57474293a65b1f5d1016a724 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:06.493247 21796 raft_consensus.cc:729] T 7aaebde43b554d51aff8c8489e6178d3 P 47327cfe57474293a65b1f5d1016a724 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47327cfe57474293a65b1f5d1016a724, State: Initialized, Role: FOLLOWER
I20260810 07:15:06.493361 21796 consensus_queue.cc:260] T 7aaebde43b554d51aff8c8489e6178d3 P 47327cfe57474293a65b1f5d1016a724 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47327cfe57474293a65b1f5d1016a724" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 44957 } } peers { permanent_uuid: "34610ddda6f7449a9cd55b3310457b3e" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 38067 } } peers { permanent_uuid: "8862807f605e42c9b2fb5277d15695a1" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33497 } }
I20260810 07:15:06.493615 21796 ts_tablet_manager.cc:1428] T 7aaebde43b554d51aff8c8489e6178d3 P 47327cfe57474293a65b1f5d1016a724: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:15:06.493676 21502 heartbeater.cc:503] Master 127.15.217.126:43801 was elected leader, sending a full tablet report...
W20260810 07:15:06.578504 21639 tablet.cc:2369] T 7aaebde43b554d51aff8c8489e6178d3 P 8862807f605e42c9b2fb5277d15695a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:15:06.702992 21780 tablet.cc:2369] T 7aaebde43b554d51aff8c8489e6178d3 P 34610ddda6f7449a9cd55b3310457b3e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:15:06.707617 21503 tablet.cc:2369] T 7aaebde43b554d51aff8c8489e6178d3 P 47327cfe57474293a65b1f5d1016a724: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:15:06.758388 21802 raft_consensus.cc:482] T 7aaebde43b554d51aff8c8489e6178d3 P 47327cfe57474293a65b1f5d1016a724 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:15:06.758606 21802 raft_consensus.cc:504] T 7aaebde43b554d51aff8c8489e6178d3 P 47327cfe57474293a65b1f5d1016a724 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47327cfe57474293a65b1f5d1016a724" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 44957 } } peers { permanent_uuid: "34610ddda6f7449a9cd55b3310457b3e" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 38067 } } peers { permanent_uuid: "8862807f605e42c9b2fb5277d15695a1" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33497 } }
I20260810 07:15:06.759074 21802 leader_election.cc:288] T 7aaebde43b554d51aff8c8489e6178d3 P 47327cfe57474293a65b1f5d1016a724 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34610ddda6f7449a9cd55b3310457b3e (127.15.217.67:38067), 8862807f605e42c9b2fb5277d15695a1 (127.15.217.66:33497)
I20260810 07:15:06.760726 21790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:06.759420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:36551 (local address 127.15.217.66:33497)
0810 07:15:06.759509 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:06.759511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:06.759523 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:15:06.759643 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:06.759644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:06.759650 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:06.759711 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:06.760416 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:06.760418 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:06.760418 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:06.760424 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:06.760440 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:06.760445 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:06.760543 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:06.760628 (+ 85us) server_negotiation.cc:299] Negotiation successful
0810 07:15:06.760646 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 07:15:06.760890 21788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:06.759396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:33661 (local address 127.15.217.67:38067)
0810 07:15:06.759454 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:06.759457 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:06.759568 (+ 111us) server_negotiation.cc:407] Connection header received
0810 07:15:06.759639 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:06.759640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:06.759665 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:06.759737 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:06.760641 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:06.760643 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:06.760643 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:06.760654 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:06.760677 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:06.760681 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:06.760752 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:06.760800 (+ 48us) server_negotiation.cc:299] Negotiation successful
0810 07:15:06.760816 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 07:15:06.760910 21803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:06.759344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:38067 (local address 127.15.217.65:33661)
0810 07:15:06.759479 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:06.759490 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:06.759575 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:06.759744 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:06.759745 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:06.759767 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:06.760537 (+ 770us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:15:06.760538 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:06.760546 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:15:06.760560 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:06.760748 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:06.760749 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:06.760750 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 07:15:06.760802 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 07:15:06.760816 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":69,"threads_started":1}
I20260810 07:15:06.761030 21805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:06.759325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:33497 (local address 127.15.217.65:36551)
0810 07:15:06.759462 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:06.759474 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:06.759534 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:06.759717 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:06.759718 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:06.759722 (+ 4us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:06.760248 (+ 526us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:15:06.760249 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:06.760258 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:15:06.760274 (+ 16us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:06.760541 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:06.760542 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:06.760543 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 07:15:06.760837 (+ 294us) client_negotiation.cc:240] Negotiation successful
0810 07:15:06.760855 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":64,"threads_started":1}
I20260810 07:15:06.761452 21592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7aaebde43b554d51aff8c8489e6178d3" candidate_uuid: "47327cfe57474293a65b1f5d1016a724" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8862807f605e42c9b2fb5277d15695a1" is_pre_election: true
I20260810 07:15:06.761445 21730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7aaebde43b554d51aff8c8489e6178d3" candidate_uuid: "47327cfe57474293a65b1f5d1016a724" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34610ddda6f7449a9cd55b3310457b3e" is_pre_election: true
I20260810 07:15:06.761641 21730 raft_consensus.cc:2454] T 7aaebde43b554d51aff8c8489e6178d3 P 34610ddda6f7449a9cd55b3310457b3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47327cfe57474293a65b1f5d1016a724 in term 0.
I20260810 07:15:06.761641 21592 raft_consensus.cc:2454] T 7aaebde43b554d51aff8c8489e6178d3 P 8862807f605e42c9b2fb5277d15695a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47327cfe57474293a65b1f5d1016a724 in term 0.
I20260810 07:15:06.761941 21380 leader_election.cc:302] T 7aaebde43b554d51aff8c8489e6178d3 P 47327cfe57474293a65b1f5d1016a724 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34610ddda6f7449a9cd55b3310457b3e, 47327cfe57474293a65b1f5d1016a724; no voters:
I20260810 07:15:06.762130 21802 raft_consensus.cc:2781] T 7aaebde43b554d51aff8c8489e6178d3 P 47327cfe57474293a65b1f5d1016a724 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:15:06.762172 21802 raft_consensus.cc:482] T 7aaebde43b554d51aff8c8489e6178d3 P 47327cfe57474293a65b1f5d1016a724 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:15:06.762188 21802 raft_consensus.cc:3037] T 7aaebde43b554d51aff8c8489e6178d3 P 47327cfe57474293a65b1f5d1016a724 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:06.762960 21802 raft_consensus.cc:504] T 7aaebde43b554d51aff8c8489e6178d3 P 47327cfe57474293a65b1f5d1016a724 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47327cfe57474293a65b1f5d1016a724" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 44957 } } peers { permanent_uuid: "34610ddda6f7449a9cd55b3310457b3e" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 38067 } } peers { permanent_uuid: "8862807f605e42c9b2fb5277d15695a1" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33497 } }
I20260810 07:15:06.763125 21802 leader_election.cc:288] T 7aaebde43b554d51aff8c8489e6178d3 P 47327cfe57474293a65b1f5d1016a724 [CANDIDATE]: Term 1 election: Requested vote from peers 34610ddda6f7449a9cd55b3310457b3e (127.15.217.67:38067), 8862807f605e42c9b2fb5277d15695a1 (127.15.217.66:33497)
I20260810 07:15:06.763321 21730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7aaebde43b554d51aff8c8489e6178d3" candidate_uuid: "47327cfe57474293a65b1f5d1016a724" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34610ddda6f7449a9cd55b3310457b3e"
I20260810 07:15:06.763403 21730 raft_consensus.cc:3037] T 7aaebde43b554d51aff8c8489e6178d3 P 34610ddda6f7449a9cd55b3310457b3e [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:06.764119 21730 raft_consensus.cc:2454] T 7aaebde43b554d51aff8c8489e6178d3 P 34610ddda6f7449a9cd55b3310457b3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47327cfe57474293a65b1f5d1016a724 in term 1.
I20260810 07:15:06.764250 21592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7aaebde43b554d51aff8c8489e6178d3" candidate_uuid: "47327cfe57474293a65b1f5d1016a724" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8862807f605e42c9b2fb5277d15695a1"
I20260810 07:15:06.764285 21380 leader_election.cc:302] T 7aaebde43b554d51aff8c8489e6178d3 P 47327cfe57474293a65b1f5d1016a724 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34610ddda6f7449a9cd55b3310457b3e, 47327cfe57474293a65b1f5d1016a724; no voters:
I20260810 07:15:06.764338 21592 raft_consensus.cc:3037] T 7aaebde43b554d51aff8c8489e6178d3 P 8862807f605e42c9b2fb5277d15695a1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:06.764396 21802 raft_consensus.cc:2781] T 7aaebde43b554d51aff8c8489e6178d3 P 47327cfe57474293a65b1f5d1016a724 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:06.764583 21802 raft_consensus.cc:686] T 7aaebde43b554d51aff8c8489e6178d3 P 47327cfe57474293a65b1f5d1016a724 [term 1 LEADER]: Becoming Leader. State: Replica: 47327cfe57474293a65b1f5d1016a724, State: Running, Role: LEADER
I20260810 07:15:06.764714 21802 consensus_queue.cc:237] T 7aaebde43b554d51aff8c8489e6178d3 P 47327cfe57474293a65b1f5d1016a724 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47327cfe57474293a65b1f5d1016a724" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 44957 } } peers { permanent_uuid: "34610ddda6f7449a9cd55b3310457b3e" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 38067 } } peers { permanent_uuid: "8862807f605e42c9b2fb5277d15695a1" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33497 } }
I20260810 07:15:06.765329 21592 raft_consensus.cc:2454] T 7aaebde43b554d51aff8c8489e6178d3 P 8862807f605e42c9b2fb5277d15695a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47327cfe57474293a65b1f5d1016a724 in term 1.
I20260810 07:15:06.765682 21296 catalog_manager.cc:5495] T 7aaebde43b554d51aff8c8489e6178d3 P 47327cfe57474293a65b1f5d1016a724 reported cstate change: term changed from 0 to 1, leader changed from <none> to 47327cfe57474293a65b1f5d1016a724 (127.15.217.65). New cstate: current_term: 1 leader_uuid: "47327cfe57474293a65b1f5d1016a724" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47327cfe57474293a65b1f5d1016a724" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 44957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34610ddda6f7449a9cd55b3310457b3e" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 38067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8862807f605e42c9b2fb5277d15695a1" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33497 } health_report { overall_health: UNKNOWN } } }
I20260810 07:15:06.833843 21789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:06.832955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49100 (local address 127.15.217.65:44957)
0810 07:15:06.833019 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:06.833021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:06.833031 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:15:06.833081 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:06.833082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:06.833097 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:06.833145 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:06.833634 (+ 489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:06.833635 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:06.833636 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:06.833662 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:06.833678 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:06.833682 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:06.833741 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:06.833773 (+ 32us) server_negotiation.cc:299] Negotiation successful
0810 07:15:06.833784 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 07:15:06.836150 21730 raft_consensus.cc:1261] T 7aaebde43b554d51aff8c8489e6178d3 P 34610ddda6f7449a9cd55b3310457b3e [term 1 FOLLOWER]: Refusing update from remote peer 47327cfe57474293a65b1f5d1016a724: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:06.836165 21592 raft_consensus.cc:1261] T 7aaebde43b554d51aff8c8489e6178d3 P 8862807f605e42c9b2fb5277d15695a1 [term 1 FOLLOWER]: Refusing update from remote peer 47327cfe57474293a65b1f5d1016a724: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:06.836545 21802 consensus_queue.cc:1035] T 7aaebde43b554d51aff8c8489e6178d3 P 47327cfe57474293a65b1f5d1016a724 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34610ddda6f7449a9cd55b3310457b3e" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 38067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:06.836737 21807 consensus_queue.cc:1035] T 7aaebde43b554d51aff8c8489e6178d3 P 47327cfe57474293a65b1f5d1016a724 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8862807f605e42c9b2fb5277d15695a1" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:06.839064 21810 mvcc.cc:205] Tried to move back new op lower bound from 7316873653597020160 to 7316873653308989440. Current Snapshot: MvccSnapshot[applied={T|T < 7316873653597020160}]
I20260810 07:15:06.839414 21811 mvcc.cc:205] Tried to move back new op lower bound from 7316873653597020160 to 7316873653308989440. Current Snapshot: MvccSnapshot[applied={T|T < 7316873653597020160}]
I20260810 07:15:06.840289 21812 mvcc.cc:205] Tried to move back new op lower bound from 7316873653597020160 to 7316873653308989440. Current Snapshot: MvccSnapshot[applied={T|T < 7316873653597020160}]
I20260810 07:15:06.841588 21297 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53424:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:06.841733 21297 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53424:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:06.842509 21297 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7fe96d3f035a4fd9af2505e033283fdb: Sending DeleteTablet for 3 replicas of tablet 7aaebde43b554d51aff8c8489e6178d3
I20260810 07:15:06.842834 21430 tablet_service.cc:1514] Processing DeleteTablet for tablet 7aaebde43b554d51aff8c8489e6178d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:06 UTC) from {username='slave'} at 127.0.0.1:49086
I20260810 07:15:06.842878 21570 tablet_service.cc:1514] Processing DeleteTablet for tablet 7aaebde43b554d51aff8c8489e6178d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:06 UTC) from {username='slave'} at 127.0.0.1:33134
I20260810 07:15:06.843082 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 21362
I20260810 07:15:06.843219 21709 tablet_service.cc:1514] Processing DeleteTablet for tablet 7aaebde43b554d51aff8c8489e6178d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:06 UTC) from {username='slave'} at 127.0.0.1:59248
I20260810 07:15:06.846472 21822 tablet_replica.cc:330] T 7aaebde43b554d51aff8c8489e6178d3 P 8862807f605e42c9b2fb5277d15695a1: stopping tablet replica
I20260810 07:15:06.846556 21822 raft_consensus.cc:2229] T 7aaebde43b554d51aff8c8489e6178d3 P 8862807f605e42c9b2fb5277d15695a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:06.846642 21822 raft_consensus.cc:2258] T 7aaebde43b554d51aff8c8489e6178d3 P 8862807f605e42c9b2fb5277d15695a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:06.846796 21823 tablet_replica.cc:330] T 7aaebde43b554d51aff8c8489e6178d3 P 34610ddda6f7449a9cd55b3310457b3e: stopping tablet replica
I20260810 07:15:06.846891 21823 raft_consensus.cc:2229] T 7aaebde43b554d51aff8c8489e6178d3 P 34610ddda6f7449a9cd55b3310457b3e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:06.846993 21823 raft_consensus.cc:2258] T 7aaebde43b554d51aff8c8489e6178d3 P 34610ddda6f7449a9cd55b3310457b3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:06.847227 21822 ts_tablet_manager.cc:1905] T 7aaebde43b554d51aff8c8489e6178d3 P 8862807f605e42c9b2fb5277d15695a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:06.847713 21823 ts_tablet_manager.cc:1905] T 7aaebde43b554d51aff8c8489e6178d3 P 34610ddda6f7449a9cd55b3310457b3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:06.849236 21822 ts_tablet_manager.cc:1918] T 7aaebde43b554d51aff8c8489e6178d3 P 8862807f605e42c9b2fb5277d15695a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:15:06.849313 21822 log.cc:1196] T 7aaebde43b554d51aff8c8489e6178d3 P 8862807f605e42c9b2fb5277d15695a1: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-1/wal/wals/7aaebde43b554d51aff8c8489e6178d3
I20260810 07:15:06.849548 21823 ts_tablet_manager.cc:1918] T 7aaebde43b554d51aff8c8489e6178d3 P 34610ddda6f7449a9cd55b3310457b3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:15:06.849592 21822 ts_tablet_manager.cc:1939] T 7aaebde43b554d51aff8c8489e6178d3 P 8862807f605e42c9b2fb5277d15695a1: Deleting consensus metadata
I20260810 07:15:06.849668 21823 log.cc:1196] T 7aaebde43b554d51aff8c8489e6178d3 P 34610ddda6f7449a9cd55b3310457b3e: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346081759552-16229-0/minicluster-data/ts-2/wal/wals/7aaebde43b554d51aff8c8489e6178d3
I20260810 07:15:06.850045 21823 ts_tablet_manager.cc:1939] T 7aaebde43b554d51aff8c8489e6178d3 P 34610ddda6f7449a9cd55b3310457b3e: Deleting consensus metadata
I20260810 07:15:06.850073 21282 catalog_manager.cc:4841] TS 8862807f605e42c9b2fb5277d15695a1 (127.15.217.66:33497): tablet 7aaebde43b554d51aff8c8489e6178d3 (table test-table [id=feec631940b34a259b83ee0d02a6e4d0]) successfully deleted
I20260810 07:15:06.850512 21283 catalog_manager.cc:4841] TS 34610ddda6f7449a9cd55b3310457b3e (127.15.217.67:38067): tablet 7aaebde43b554d51aff8c8489e6178d3 (table test-table [id=feec631940b34a259b83ee0d02a6e4d0]) successfully deleted
W20260810 07:15:06.854842 21281 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.217.65:44957 (error 108)
I20260810 07:15:06.855052 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 21505
I20260810 07:15:06.856096 21787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:06.855248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:44957 (local address 127.0.0.1:49112)
0810 07:15:06.856011 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:06.856065 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.65:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":741}
W20260810 07:15:06.856238 21281 catalog_manager.cc:4584] TS 47327cfe57474293a65b1f5d1016a724 (127.15.217.65:44957): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7aaebde43b554d51aff8c8489e6178d3: Network error: Client connection negotiation failed: client connection to 127.15.217.65:44957: connect: Connection refused (error 111)
I20260810 07:15:06.862773 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 21641
I20260810 07:15:06.870463 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 21264
2026-08-10T07:15:06Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (971 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260810 07:15:06.893340 16229 mini_oidc.cc:152] Starting JWKS server
I20260810 07:15:06.894588 16229 webserver.cc:460] Webserver started at https://127.0.0.1:34647/ using document root <none> and password file <none>
I20260810 07:15:06.894730 16229 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 07:15:06.894949 16229 webserver.cc:460] Webserver started at http://127.0.0.1:41797/ using document root <none> and password file <none>
2026-08-10T07:15:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:15:06Z Disabled control of system clock
I20260810 07:15:06.905637 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:46343
--webserver_interface=127.15.217.126
--webserver_port=0
--builtin_ntp_servers=127.15.217.84:39935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:46343
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34647/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346081759552-16229-0/testchainca.cert with env {}
W20260810 07:15:06.994571 21830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:06.994808 21830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:06.994831 21830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:06.995993 21830 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:06.996032 21830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:06.996045 21830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:06.996058 21830 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:06.996070 21830 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 07:15:06.996083 21830 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 07:15:06.996095 21830 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34647/jwks.json
I20260810 07:15:06.997710 21830 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:39935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:46343
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:46343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34647/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346081759552-16229-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:06.997936 21830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:06.998126 21830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:07.003129 21835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:07.003193 21838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:07.003203 21836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:07.003420 21830 server_base.cc:1034] running on GCE node
I20260810 07:15:07.003741 21830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:07.004004 21830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:07.005159 21830 hybrid_clock.cc:648] HybridClock initialized: now 1786346107005141 us; error 43 us; skew 500 ppm
I20260810 07:15:07.006491 21830 webserver.cc:460] Webserver started at http://127.15.217.126:32777/ using document root <none> and password file <none>
I20260810 07:15:07.006769 21830 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:07.006820 21830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:07.006982 21830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:07.008107 21830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "dd817c8eb5464d11a07903bd0603aa3b"
format_stamp: "Formatted at 2026-08-10 07:15:07 on dist-test-slave-10pc"
server_key: "87204245251e17f3d508cb15b5e14d09"
server_key_iv: "f7d17105f0ba89da2662d36fd69e6ae2"
server_key_version: "encryptionkey@0"
I20260810 07:15:07.008558 21830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "dd817c8eb5464d11a07903bd0603aa3b"
format_stamp: "Formatted at 2026-08-10 07:15:07 on dist-test-slave-10pc"
server_key: "87204245251e17f3d508cb15b5e14d09"
server_key_iv: "f7d17105f0ba89da2662d36fd69e6ae2"
server_key_version: "encryptionkey@0"
I20260810 07:15:07.009944 21830 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:15:07.010794 21845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:07.011034 21830 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:15:07.011147 21830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "dd817c8eb5464d11a07903bd0603aa3b"
format_stamp: "Formatted at 2026-08-10 07:15:07 on dist-test-slave-10pc"
server_key: "87204245251e17f3d508cb15b5e14d09"
server_key_iv: "f7d17105f0ba89da2662d36fd69e6ae2"
server_key_version: "encryptionkey@0"
I20260810 07:15:07.011227 21830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:07.021327 21830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:07.021637 21830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:07.021799 21830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:07.026114 21830 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:46343
I20260810 07:15:07.026152 21900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:46343 every 8 connection(s)
I20260810 07:15:07.026522 21830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:15:07.027485 21901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:07.029821 21901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd817c8eb5464d11a07903bd0603aa3b: Bootstrap starting.
I20260810 07:15:07.030385 21901 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd817c8eb5464d11a07903bd0603aa3b: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:07.030660 21901 log.cc:824] T 00000000000000000000000000000000 P dd817c8eb5464d11a07903bd0603aa3b: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:07.030858 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 21830
I20260810 07:15:07.030989 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:15:07.031205 16229 external_mini_cluster.cc:1428] Setting key ad0a686f0f343dd9ff22e13f9fcb6723
I20260810 07:15:07.031502 21901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd817c8eb5464d11a07903bd0603aa3b: No bootstrap required, opened a new log
I20260810 07:15:07.033368 21901 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd817c8eb5464d11a07903bd0603aa3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd817c8eb5464d11a07903bd0603aa3b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 46343 } }
I20260810 07:15:07.033525 21901 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd817c8eb5464d11a07903bd0603aa3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:07.033622 21901 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd817c8eb5464d11a07903bd0603aa3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd817c8eb5464d11a07903bd0603aa3b, State: Initialized, Role: FOLLOWER
I20260810 07:15:07.033800 21901 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd817c8eb5464d11a07903bd0603aa3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd817c8eb5464d11a07903bd0603aa3b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 46343 } }
I20260810 07:15:07.033890 21901 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd817c8eb5464d11a07903bd0603aa3b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:15:07.033924 21901 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd817c8eb5464d11a07903bd0603aa3b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:15:07.033975 21901 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd817c8eb5464d11a07903bd0603aa3b [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:07.034730 21901 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd817c8eb5464d11a07903bd0603aa3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd817c8eb5464d11a07903bd0603aa3b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 46343 } }
I20260810 07:15:07.034842 21901 leader_election.cc:302] T 00000000000000000000000000000000 P dd817c8eb5464d11a07903bd0603aa3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd817c8eb5464d11a07903bd0603aa3b; no voters:
I20260810 07:15:07.034992 21901 leader_election.cc:288] T 00000000000000000000000000000000 P dd817c8eb5464d11a07903bd0603aa3b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:15:07.035029 21907 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd817c8eb5464d11a07903bd0603aa3b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:07.035214 21907 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd817c8eb5464d11a07903bd0603aa3b [term 1 LEADER]: Becoming Leader. State: Replica: dd817c8eb5464d11a07903bd0603aa3b, State: Running, Role: LEADER
I20260810 07:15:07.035240 21901 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd817c8eb5464d11a07903bd0603aa3b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:07.035365 21907 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd817c8eb5464d11a07903bd0603aa3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd817c8eb5464d11a07903bd0603aa3b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 46343 } }
I20260810 07:15:07.035687 21907 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd817c8eb5464d11a07903bd0603aa3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd817c8eb5464d11a07903bd0603aa3b. Latest consensus state: current_term: 1 leader_uuid: "dd817c8eb5464d11a07903bd0603aa3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd817c8eb5464d11a07903bd0603aa3b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 46343 } } }
I20260810 07:15:07.035784 21907 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd817c8eb5464d11a07903bd0603aa3b [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:07.035703 21909 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd817c8eb5464d11a07903bd0603aa3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd817c8eb5464d11a07903bd0603aa3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd817c8eb5464d11a07903bd0603aa3b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 46343 } } }
I20260810 07:15:07.035926 21909 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd817c8eb5464d11a07903bd0603aa3b [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:07.036098 21916 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:07.036729 21916 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:15:07.039140 21906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.031687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57940 (local address 127.15.217.126:46343)
0810 07:15:07.031914 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:07.031922 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:07.031961 (+ 39us) server_negotiation.cc:407] Connection header received
0810 07:15:07.033352 (+ 1391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:07.033359 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:07.033425 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:07.033575 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:07.033858 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.035675 (+ 1817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.036964 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.037594 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.037719 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.038462 (+ 743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:07.038467 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:07.038467 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:07.038653 (+ 186us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:07.038686 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:07.038696 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:07.038774 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:07.038852 (+ 78us) server_negotiation.cc:299] Negotiation successful
0810 07:15:07.038933 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":77,"threads_started":1}
I20260810 07:15:07.040202 16229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:15:07.040407 21916 catalog_manager.cc:1347] Generated new cluster ID: 971eaa636813445299f1bf85f5393745
I20260810 07:15:07.040486 21916 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:15:07.053211 21916 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:15:07.053969 21916 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:07.058904 21916 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd817c8eb5464d11a07903bd0603aa3b: Generated new TSK 0
I20260810 07:15:07.059134 21916 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 07:15:07.094672 21936 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260810 07:15:07.094835 21936 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 07:15:07.094938 21906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.072045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57950 (local address 127.15.217.126:46343)
0810 07:15:07.072103 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:07.072106 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:07.072268 (+ 162us) server_negotiation.cc:407] Connection header received
0810 07:15:07.072381 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:07.072382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:07.072402 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:07.072492 (+ 90us) server_negotiation.cc:226] Negotiated authn=JWT
0810 07:15:07.072681 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.073178 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.073955 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.074148 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.074229 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.074471 (+ 242us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 07:15:07.094621 (+ 20150us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34647/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 07:15:07.094861 (+ 240us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57950: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34647/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":33}
W20260810 07:15:07.095014 21906 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57950: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34647/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260810 07:15:07.095656 21935 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:07.071963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:46343 (local address 127.0.0.1:57950)
0810 07:15:07.072197 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:07.072207 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:07.072292 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:07.072508 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:07.072510 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:07.072512 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0810 07:15:07.072589 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:07.072592 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.073266 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.073267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:07.073861 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:07.073863 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.074241 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.074243 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:07.074306 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.074334 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 07:15:07.094985 (+ 20651us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34647/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 07:15:07.095316 (+ 331us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.217.126:46343: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34647/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":90,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.15.217.126:46343: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34647/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260810 07:15:07.096164 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 21830
2026-08-10T07:15:07Z chronyd exiting
I20260810 07:15:07.137756 16229 test_util.cc:175] -----------------------------------------------
I20260810 07:15:07.137849 16229 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346081759552-16229-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (245 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260810 07:15:07.138566 16229 mini_oidc.cc:152] Starting JWKS server
I20260810 07:15:07.139237 16229 webserver.cc:460] Webserver started at https://127.0.0.1:39053/ using document root <none> and password file <none>
I20260810 07:15:07.139310 16229 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 07:15:07.139488 16229 webserver.cc:460] Webserver started at http://127.0.0.1:40499/ using document root <none> and password file <none>
2026-08-10T07:15:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:15:07Z Disabled control of system clock
I20260810 07:15:07.148722 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:41017
--webserver_interface=127.15.217.126
--webserver_port=0
--builtin_ntp_servers=127.15.217.84:40373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:41017
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39053/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346081759552-16229-0/testchainca.cert with env {}
W20260810 07:15:07.234102 21942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:07.234318 21942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:07.234339 21942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:07.235436 21942 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:07.235471 21942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:07.235484 21942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:07.235497 21942 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:07.235509 21942 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 07:15:07.235522 21942 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 07:15:07.235532 21942 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39053/jwks.json
I20260810 07:15:07.237071 21942 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:40373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:41017
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:41017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39053/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346081759552-16229-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:07.237336 21942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:07.237603 21942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:07.242784 21949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:07.242972 21942 server_base.cc:1034] running on GCE node
W20260810 07:15:07.242784 21947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:07.242782 21951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:07.243294 21942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:07.243609 21942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:07.244745 21942 hybrid_clock.cc:648] HybridClock initialized: now 1786346107244737 us; error 37 us; skew 500 ppm
I20260810 07:15:07.246040 21942 webserver.cc:460] Webserver started at http://127.15.217.126:36783/ using document root <none> and password file <none>
I20260810 07:15:07.246301 21942 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:07.246351 21942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:07.246523 21942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:07.247661 21942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "c600e1a33cb64d3eb0e9331af37ae40b"
format_stamp: "Formatted at 2026-08-10 07:15:07 on dist-test-slave-10pc"
server_key: "b4bd316e42e6cd33471dd5074eaa943c"
server_key_iv: "83a41e1d55f6be7ca8ba33ffe124c30c"
server_key_version: "encryptionkey@0"
I20260810 07:15:07.248056 21942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "c600e1a33cb64d3eb0e9331af37ae40b"
format_stamp: "Formatted at 2026-08-10 07:15:07 on dist-test-slave-10pc"
server_key: "b4bd316e42e6cd33471dd5074eaa943c"
server_key_iv: "83a41e1d55f6be7ca8ba33ffe124c30c"
server_key_version: "encryptionkey@0"
I20260810 07:15:07.249526 21942 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:15:07.250293 21957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:07.250471 21942 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 07:15:07.250583 21942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "c600e1a33cb64d3eb0e9331af37ae40b"
format_stamp: "Formatted at 2026-08-10 07:15:07 on dist-test-slave-10pc"
server_key: "b4bd316e42e6cd33471dd5074eaa943c"
server_key_iv: "83a41e1d55f6be7ca8ba33ffe124c30c"
server_key_version: "encryptionkey@0"
I20260810 07:15:07.250663 21942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:07.271654 21942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:07.272051 21942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:07.272230 21942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:07.276610 21942 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:41017
I20260810 07:15:07.276683 22013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:41017 every 8 connection(s)
I20260810 07:15:07.277068 21942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:15:07.277796 22014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:07.280967 22014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c600e1a33cb64d3eb0e9331af37ae40b: Bootstrap starting.
I20260810 07:15:07.281528 22014 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c600e1a33cb64d3eb0e9331af37ae40b: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:07.281788 22014 log.cc:824] T 00000000000000000000000000000000 P c600e1a33cb64d3eb0e9331af37ae40b: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:07.282493 22014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c600e1a33cb64d3eb0e9331af37ae40b: No bootstrap required, opened a new log
I20260810 07:15:07.283592 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 21942
I20260810 07:15:07.283712 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:15:07.283905 16229 external_mini_cluster.cc:1428] Setting key 9e971b4468cce7196d37ff2d6480be16
I20260810 07:15:07.284765 22014 raft_consensus.cc:348] T 00000000000000000000000000000000 P c600e1a33cb64d3eb0e9331af37ae40b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c600e1a33cb64d3eb0e9331af37ae40b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41017 } }
I20260810 07:15:07.284914 22014 raft_consensus.cc:374] T 00000000000000000000000000000000 P c600e1a33cb64d3eb0e9331af37ae40b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:07.284951 22014 raft_consensus.cc:729] T 00000000000000000000000000000000 P c600e1a33cb64d3eb0e9331af37ae40b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c600e1a33cb64d3eb0e9331af37ae40b, State: Initialized, Role: FOLLOWER
I20260810 07:15:07.285120 22014 consensus_queue.cc:260] T 00000000000000000000000000000000 P c600e1a33cb64d3eb0e9331af37ae40b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c600e1a33cb64d3eb0e9331af37ae40b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41017 } }
I20260810 07:15:07.285223 22014 raft_consensus.cc:388] T 00000000000000000000000000000000 P c600e1a33cb64d3eb0e9331af37ae40b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:15:07.285272 22014 raft_consensus.cc:482] T 00000000000000000000000000000000 P c600e1a33cb64d3eb0e9331af37ae40b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:15:07.285332 22014 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c600e1a33cb64d3eb0e9331af37ae40b [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:07.285907 22014 raft_consensus.cc:504] T 00000000000000000000000000000000 P c600e1a33cb64d3eb0e9331af37ae40b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c600e1a33cb64d3eb0e9331af37ae40b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41017 } }
I20260810 07:15:07.285997 22014 leader_election.cc:302] T 00000000000000000000000000000000 P c600e1a33cb64d3eb0e9331af37ae40b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c600e1a33cb64d3eb0e9331af37ae40b; no voters:
I20260810 07:15:07.286234 22014 leader_election.cc:288] T 00000000000000000000000000000000 P c600e1a33cb64d3eb0e9331af37ae40b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:15:07.286255 22022 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c600e1a33cb64d3eb0e9331af37ae40b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:07.286372 22022 raft_consensus.cc:686] T 00000000000000000000000000000000 P c600e1a33cb64d3eb0e9331af37ae40b [term 1 LEADER]: Becoming Leader. State: Replica: c600e1a33cb64d3eb0e9331af37ae40b, State: Running, Role: LEADER
I20260810 07:15:07.286466 22022 consensus_queue.cc:237] T 00000000000000000000000000000000 P c600e1a33cb64d3eb0e9331af37ae40b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c600e1a33cb64d3eb0e9331af37ae40b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41017 } }
I20260810 07:15:07.286522 22014 sys_catalog.cc:564] T 00000000000000000000000000000000 P c600e1a33cb64d3eb0e9331af37ae40b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:07.286789 22024 sys_catalog.cc:455] T 00000000000000000000000000000000 P c600e1a33cb64d3eb0e9331af37ae40b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c600e1a33cb64d3eb0e9331af37ae40b. Latest consensus state: current_term: 1 leader_uuid: "c600e1a33cb64d3eb0e9331af37ae40b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c600e1a33cb64d3eb0e9331af37ae40b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41017 } } }
I20260810 07:15:07.286981 22024 sys_catalog.cc:458] T 00000000000000000000000000000000 P c600e1a33cb64d3eb0e9331af37ae40b [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:07.286924 22023 sys_catalog.cc:455] T 00000000000000000000000000000000 P c600e1a33cb64d3eb0e9331af37ae40b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c600e1a33cb64d3eb0e9331af37ae40b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c600e1a33cb64d3eb0e9331af37ae40b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41017 } } }
I20260810 07:15:07.287277 22023 sys_catalog.cc:458] T 00000000000000000000000000000000 P c600e1a33cb64d3eb0e9331af37ae40b [sys.catalog]: This master's current role is: LEADER
W20260810 07:15:07.288424 22040 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c600e1a33cb64d3eb0e9331af37ae40b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:15:07.288499 22040 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:15:07.288944 22032 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:07.289155 22032 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:15:07.290477 22021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.284491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45686 (local address 127.15.217.126:41017)
0810 07:15:07.284724 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:07.284732 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:07.284758 (+ 26us) server_negotiation.cc:407] Connection header received
0810 07:15:07.284858 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:07.284863 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:07.284952 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:07.285148 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:07.285365 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.287030 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.288113 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.288673 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.288768 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.289646 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:07.289650 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:07.289651 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:07.289868 (+ 217us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:07.289921 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:07.289933 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:07.290022 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:07.290103 (+ 81us) server_negotiation.cc:299] Negotiation successful
0810 07:15:07.290204 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":109,"threads_started":1}
I20260810 07:15:07.291115 22032 catalog_manager.cc:1347] Generated new cluster ID: 9608e24ea55044978a6511dee69f4d0b
I20260810 07:15:07.291214 22032 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:15:07.291589 16229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:15:07.320379 22032 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:15:07.321034 22032 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:07.327042 22032 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c600e1a33cb64d3eb0e9331af37ae40b: Generated new TSK 0
I20260810 07:15:07.327248 22032 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 07:15:07.353793 22051 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260810 07:15:07.353910 22051 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 07:15:07.354279 22021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.339622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45694 (local address 127.15.217.126:41017)
0810 07:15:07.339691 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:07.339694 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:07.339828 (+ 134us) server_negotiation.cc:407] Connection header received
0810 07:15:07.339916 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:07.339917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:07.339931 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:07.340019 (+ 88us) server_negotiation.cc:226] Negotiated authn=JWT
0810 07:15:07.340156 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.340964 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.341777 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.342084 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.342159 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.342339 (+ 180us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 07:15:07.354025 (+ 11686us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39053/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 07:15:07.354179 (+ 154us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45694: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39053/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":38}
W20260810 07:15:07.354341 22021 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45694: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39053/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260810 07:15:07.354431 22050 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:07.339578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41017 (local address 127.0.0.1:45694)
0810 07:15:07.339765 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:07.339775 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:07.339844 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:07.340025 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:07.340026 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:07.340028 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0810 07:15:07.340088 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:07.340090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.341046 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.341047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:07.341684 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:07.341687 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.342175 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.342176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:07.342223 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.342229 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 07:15:07.354160 (+ 11931us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39053/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 07:15:07.354288 (+ 128us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.217.126:41017: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39053/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":134,"threads_started":1}
I20260810 07:15:07.354830 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 21942
2026-08-10T07:15:07Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (257 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260810 07:15:07.396061 16229 mini_oidc.cc:152] Starting JWKS server
I20260810 07:15:07.396895 16229 webserver.cc:460] Webserver started at https://127.0.0.1:37147/ using document root <none> and password file <none>
I20260810 07:15:07.396947 16229 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 07:15:07.397118 16229 webserver.cc:460] Webserver started at http://127.0.0.1:35359/ using document root <none> and password file <none>
2026-08-10T07:15:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:15:07Z Disabled control of system clock
I20260810 07:15:07.406948 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:41293
--webserver_interface=127.15.217.126
--webserver_port=0
--builtin_ntp_servers=127.15.217.84:43435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:41293
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37147/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 07:15:07.493251 22058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:07.493459 22058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:07.493482 22058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:07.494583 22058 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:07.494618 22058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:07.494632 22058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:07.494647 22058 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:07.494658 22058 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 07:15:07.494670 22058 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 07:15:07.494683 22058 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37147/jwks.json
I20260810 07:15:07.495941 22058 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:43435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:41293
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:41293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37147/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:07.496153 22058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:07.496398 22058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:07.501808 22065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:07.501842 22067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:07.501991 22058 server_base.cc:1034] running on GCE node
W20260810 07:15:07.502104 22064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:07.502332 22058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:07.502647 22058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:07.503911 22058 hybrid_clock.cc:648] HybridClock initialized: now 1786346107503896 us; error 41 us; skew 500 ppm
I20260810 07:15:07.505342 22058 webserver.cc:460] Webserver started at http://127.15.217.126:38839/ using document root <none> and password file <none>
I20260810 07:15:07.505618 22058 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:07.505692 22058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:07.505856 22058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:07.507277 22058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "59f93fe1e25d49829cf49823cacaf734"
format_stamp: "Formatted at 2026-08-10 07:15:07 on dist-test-slave-10pc"
server_key: "dea7ee7e7ff817aa9d8474b17f2d49d8"
server_key_iv: "88af83ff6ef2db02fd77ed5cc26387f1"
server_key_version: "encryptionkey@0"
I20260810 07:15:07.507716 22058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "59f93fe1e25d49829cf49823cacaf734"
format_stamp: "Formatted at 2026-08-10 07:15:07 on dist-test-slave-10pc"
server_key: "dea7ee7e7ff817aa9d8474b17f2d49d8"
server_key_iv: "88af83ff6ef2db02fd77ed5cc26387f1"
server_key_version: "encryptionkey@0"
I20260810 07:15:07.509357 22058 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260810 07:15:07.510133 22074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:07.510326 22058 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 07:15:07.510444 22058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "59f93fe1e25d49829cf49823cacaf734"
format_stamp: "Formatted at 2026-08-10 07:15:07 on dist-test-slave-10pc"
server_key: "dea7ee7e7ff817aa9d8474b17f2d49d8"
server_key_iv: "88af83ff6ef2db02fd77ed5cc26387f1"
server_key_version: "encryptionkey@0"
I20260810 07:15:07.510527 22058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:07.534086 22058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:07.534495 22058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:07.534673 22058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:07.539389 22058 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:41293
I20260810 07:15:07.539427 22128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:41293 every 8 connection(s)
I20260810 07:15:07.539820 22058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:15:07.540576 22130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:07.542294 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 22058
I20260810 07:15:07.542425 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:15:07.542621 16229 external_mini_cluster.cc:1428] Setting key f48dc45455d23d80b7ae5e9b550763f2
I20260810 07:15:07.543936 22130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59f93fe1e25d49829cf49823cacaf734: Bootstrap starting.
I20260810 07:15:07.544633 22130 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59f93fe1e25d49829cf49823cacaf734: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:07.544986 22130 log.cc:824] T 00000000000000000000000000000000 P 59f93fe1e25d49829cf49823cacaf734: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:07.545746 22130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59f93fe1e25d49829cf49823cacaf734: No bootstrap required, opened a new log
I20260810 07:15:07.547739 22130 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59f93fe1e25d49829cf49823cacaf734 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59f93fe1e25d49829cf49823cacaf734" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41293 } }
I20260810 07:15:07.547909 22130 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59f93fe1e25d49829cf49823cacaf734 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:07.547966 22130 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59f93fe1e25d49829cf49823cacaf734 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59f93fe1e25d49829cf49823cacaf734, State: Initialized, Role: FOLLOWER
I20260810 07:15:07.548122 22130 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59f93fe1e25d49829cf49823cacaf734 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59f93fe1e25d49829cf49823cacaf734" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41293 } }
I20260810 07:15:07.548224 22130 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59f93fe1e25d49829cf49823cacaf734 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:15:07.548267 22130 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59f93fe1e25d49829cf49823cacaf734 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:15:07.548314 22130 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59f93fe1e25d49829cf49823cacaf734 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:07.548632 22133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.543210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37196 (local address 127.15.217.126:41293)
0810 07:15:07.543412 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:07.543419 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:07.543445 (+ 26us) server_negotiation.cc:407] Connection header received
0810 07:15:07.543861 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:07.543868 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:07.543937 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:07.544097 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:07.544335 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.545408 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.546265 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.546711 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.546848 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.547786 (+ 938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:07.547791 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:07.547791 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:07.547993 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:07.548041 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:07.548057 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:07.548120 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:07.548190 (+ 70us) server_negotiation.cc:299] Negotiation successful
0810 07:15:07.548372 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":85,"threads_started":1}
I20260810 07:15:07.549073 22130 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59f93fe1e25d49829cf49823cacaf734 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59f93fe1e25d49829cf49823cacaf734" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41293 } }
I20260810 07:15:07.549182 22130 leader_election.cc:302] T 00000000000000000000000000000000 P 59f93fe1e25d49829cf49823cacaf734 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59f93fe1e25d49829cf49823cacaf734; no voters:
I20260810 07:15:07.549348 22130 leader_election.cc:288] T 00000000000000000000000000000000 P 59f93fe1e25d49829cf49823cacaf734 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:15:07.549393 22136 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59f93fe1e25d49829cf49823cacaf734 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:07.549602 16229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:15:07.549562 22136 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59f93fe1e25d49829cf49823cacaf734 [term 1 LEADER]: Becoming Leader. State: Replica: 59f93fe1e25d49829cf49823cacaf734, State: Running, Role: LEADER
I20260810 07:15:07.549612 22130 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59f93fe1e25d49829cf49823cacaf734 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:07.550359 22136 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59f93fe1e25d49829cf49823cacaf734 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59f93fe1e25d49829cf49823cacaf734" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41293 } }
I20260810 07:15:07.552147 22137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59f93fe1e25d49829cf49823cacaf734 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59f93fe1e25d49829cf49823cacaf734" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59f93fe1e25d49829cf49823cacaf734" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41293 } } }
I20260810 07:15:07.552132 22138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59f93fe1e25d49829cf49823cacaf734 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59f93fe1e25d49829cf49823cacaf734. Latest consensus state: current_term: 1 leader_uuid: "59f93fe1e25d49829cf49823cacaf734" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59f93fe1e25d49829cf49823cacaf734" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41293 } } }
I20260810 07:15:07.552266 22137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59f93fe1e25d49829cf49823cacaf734 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:07.552274 22138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59f93fe1e25d49829cf49823cacaf734 [sys.catalog]: This master's current role is: LEADER
W20260810 07:15:07.552507 22153 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 59f93fe1e25d49829cf49823cacaf734: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:15:07.552567 22153 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:15:07.552634 22155 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:07.552821 22155 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:15:07.554399 22155 catalog_manager.cc:1347] Generated new cluster ID: b9394f9dae1a4ce1b839c9e9a33a1ca9
I20260810 07:15:07.554458 22155 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:15:07.558946 22155 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:15:07.559729 22155 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:07.563558 22155 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59f93fe1e25d49829cf49823cacaf734: Generated new TSK 0
I20260810 07:15:07.563745 22155 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:15:07.573598 22133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.569689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37210 (local address 127.15.217.126:41293)
0810 07:15:07.569760 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:07.569762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:07.569896 (+ 134us) server_negotiation.cc:407] Connection header received
0810 07:15:07.569941 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:07.569942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:07.569954 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:07.570034 (+ 80us) server_negotiation.cc:226] Negotiated authn=JWT
0810 07:15:07.570155 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.570935 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.571740 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.572084 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.572159 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.572334 (+ 175us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 07:15:07.573380 (+ 1046us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37147/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 07:15:07.573525 (+ 145us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37210: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37147/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":42}
W20260810 07:15:07.573673 22133 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37210: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37147/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260810 07:15:07.575541 22162 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:07.569617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41293 (local address 127.0.0.1:37210)
0810 07:15:07.569816 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:07.569833 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:07.569889 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:07.570040 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:07.570040 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:07.570042 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0810 07:15:07.570100 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:07.570102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.571033 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.571034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:07.571652 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:07.571654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.572181 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.572182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:07.572229 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.572235 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 07:15:07.573496 (+ 1261us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37147/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 07:15:07.575435 (+ 1939us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.217.126:41293: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37147/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":1726,"thread_start_us":131,"threads_started":1}
I20260810 07:15:07.575930 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 22058
E20260810 07:15:07.589004 22163 webserver.cc:533] Webserver: sslize failed:
E20260810 07:15:07.589077 22163 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-10T07:15:07Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (222 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-10T07:15:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:15:07Z Disabled control of system clock
I20260810 07:15:07.628037 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:35945
--webserver_interface=127.15.217.126
--webserver_port=0
--builtin_ntp_servers=127.15.217.84:39987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:35945
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346081759552-16229-0/testchainca.cert with env {}
W20260810 07:15:07.712875 22168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:07.713099 22168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:07.713125 22168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:07.714395 22168 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:07.714444 22168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:07.714457 22168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:07.714470 22168 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:07.714483 22168 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 07:15:07.714496 22168 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 07:15:07.714551 22168 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260810 07:15:07.716087 22168 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:39987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:35945
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:35945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346081759552-16229-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:07.716382 22168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:07.716638 22168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:07.721693 22176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:07.721781 22173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:07.721714 22174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:07.722450 22168 server_base.cc:1034] running on GCE node
I20260810 07:15:07.722749 22168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:07.723093 22168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:07.724247 22168 hybrid_clock.cc:648] HybridClock initialized: now 1786346107724220 us; error 41 us; skew 500 ppm
I20260810 07:15:07.726287 22168 webserver.cc:460] Webserver started at http://127.15.217.126:38257/ using document root <none> and password file <none>
I20260810 07:15:07.726616 22168 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:07.726702 22168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:07.726919 22168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:07.728235 22168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "7b100c0a57aa4e25924ad0eeb2f95d0a"
format_stamp: "Formatted at 2026-08-10 07:15:07 on dist-test-slave-10pc"
server_key: "664a78004877bfee38f54e5ee27f3e6a"
server_key_iv: "792e73e65d1ede69b2beb544c651968f"
server_key_version: "encryptionkey@0"
I20260810 07:15:07.728699 22168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "7b100c0a57aa4e25924ad0eeb2f95d0a"
format_stamp: "Formatted at 2026-08-10 07:15:07 on dist-test-slave-10pc"
server_key: "664a78004877bfee38f54e5ee27f3e6a"
server_key_iv: "792e73e65d1ede69b2beb544c651968f"
server_key_version: "encryptionkey@0"
I20260810 07:15:07.730252 22168 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:15:07.731108 22183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:07.731300 22168 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:15:07.731427 22168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "7b100c0a57aa4e25924ad0eeb2f95d0a"
format_stamp: "Formatted at 2026-08-10 07:15:07 on dist-test-slave-10pc"
server_key: "664a78004877bfee38f54e5ee27f3e6a"
server_key_iv: "792e73e65d1ede69b2beb544c651968f"
server_key_version: "encryptionkey@0"
I20260810 07:15:07.731518 22168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:07.744948 22168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:07.745350 22168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:07.745537 22168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:07.750262 22168 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:35945
I20260810 07:15:07.750317 22239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:35945 every 8 connection(s)
I20260810 07:15:07.750761 22168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:15:07.751905 22240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:07.753077 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 22168
I20260810 07:15:07.753216 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:15:07.753445 16229 external_mini_cluster.cc:1428] Setting key 4c60522a625d95c412df6474c8551440
I20260810 07:15:07.755219 22240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b100c0a57aa4e25924ad0eeb2f95d0a: Bootstrap starting.
I20260810 07:15:07.756033 22240 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b100c0a57aa4e25924ad0eeb2f95d0a: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:07.756376 22240 log.cc:824] T 00000000000000000000000000000000 P 7b100c0a57aa4e25924ad0eeb2f95d0a: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:07.757349 22240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b100c0a57aa4e25924ad0eeb2f95d0a: No bootstrap required, opened a new log
I20260810 07:15:07.760016 22243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.754073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34232 (local address 127.15.217.126:35945)
0810 07:15:07.754342 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:07.754351 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:07.754381 (+ 30us) server_negotiation.cc:407] Connection header received
0810 07:15:07.755036 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:07.755045 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:07.755132 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:07.755271 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:07.755535 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.756752 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.757876 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.758305 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.758421 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.759241 (+ 820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:07.759245 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:07.759245 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:07.759425 (+ 180us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:07.759459 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:07.759491 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:07.759548 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:07.759670 (+ 122us) server_negotiation.cc:299] Negotiation successful
0810 07:15:07.759783 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":135,"threads_started":1}
I20260810 07:15:07.759967 22240 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b100c0a57aa4e25924ad0eeb2f95d0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b100c0a57aa4e25924ad0eeb2f95d0a" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35945 } }
I20260810 07:15:07.760111 22240 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b100c0a57aa4e25924ad0eeb2f95d0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:07.760170 22240 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b100c0a57aa4e25924ad0eeb2f95d0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b100c0a57aa4e25924ad0eeb2f95d0a, State: Initialized, Role: FOLLOWER
I20260810 07:15:07.760332 22240 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b100c0a57aa4e25924ad0eeb2f95d0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b100c0a57aa4e25924ad0eeb2f95d0a" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35945 } }
I20260810 07:15:07.760432 22240 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b100c0a57aa4e25924ad0eeb2f95d0a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:15:07.760521 22240 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b100c0a57aa4e25924ad0eeb2f95d0a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:15:07.760592 22240 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b100c0a57aa4e25924ad0eeb2f95d0a [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:07.761220 16229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:15:07.761507 22240 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b100c0a57aa4e25924ad0eeb2f95d0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b100c0a57aa4e25924ad0eeb2f95d0a" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35945 } }
I20260810 07:15:07.761628 22240 leader_election.cc:302] T 00000000000000000000000000000000 P 7b100c0a57aa4e25924ad0eeb2f95d0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b100c0a57aa4e25924ad0eeb2f95d0a; no voters:
I20260810 07:15:07.761830 22240 leader_election.cc:288] T 00000000000000000000000000000000 P 7b100c0a57aa4e25924ad0eeb2f95d0a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:15:07.761886 22246 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b100c0a57aa4e25924ad0eeb2f95d0a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:07.762108 22240 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b100c0a57aa4e25924ad0eeb2f95d0a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:07.762131 22246 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b100c0a57aa4e25924ad0eeb2f95d0a [term 1 LEADER]: Becoming Leader. State: Replica: 7b100c0a57aa4e25924ad0eeb2f95d0a, State: Running, Role: LEADER
I20260810 07:15:07.762301 22246 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b100c0a57aa4e25924ad0eeb2f95d0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b100c0a57aa4e25924ad0eeb2f95d0a" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35945 } }
I20260810 07:15:07.762785 22247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b100c0a57aa4e25924ad0eeb2f95d0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b100c0a57aa4e25924ad0eeb2f95d0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b100c0a57aa4e25924ad0eeb2f95d0a" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35945 } } }
I20260810 07:15:07.762799 22248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b100c0a57aa4e25924ad0eeb2f95d0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b100c0a57aa4e25924ad0eeb2f95d0a. Latest consensus state: current_term: 1 leader_uuid: "7b100c0a57aa4e25924ad0eeb2f95d0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b100c0a57aa4e25924ad0eeb2f95d0a" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35945 } } }
I20260810 07:15:07.762966 22247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b100c0a57aa4e25924ad0eeb2f95d0a [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:07.763041 22248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b100c0a57aa4e25924ad0eeb2f95d0a [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:07.763350 22256 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:07.763908 22256 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:15:07.765630 22256 catalog_manager.cc:1347] Generated new cluster ID: deef5fc6c73c41119e1ed6c57b92125a
I20260810 07:15:07.765708 22256 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:15:07.788514 22256 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:15:07.789263 22256 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:07.793044 22256 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b100c0a57aa4e25924ad0eeb2f95d0a: Generated new TSK 0
I20260810 07:15:07.793285 22256 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:15:07.866986 22243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.808243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34240 (local address 127.15.217.126:35945)
0810 07:15:07.808322 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:07.808324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:07.808500 (+ 176us) server_negotiation.cc:407] Connection header received
0810 07:15:07.808584 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:07.808585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:07.808598 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:07.808666 (+ 68us) server_negotiation.cc:226] Negotiated authn=JWT
0810 07:15:07.808823 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.809354 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.810548 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.810738 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.810796 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.811005 (+ 209us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 07:15:07.866764 (+ 55759us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 07:15:07.866924 (+ 160us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34240: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":48}
W20260810 07:15:07.867096 22243 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34240: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260810 07:15:07.867360 22271 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:07.808173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35945 (local address 127.0.0.1:34240)
0810 07:15:07.808379 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:07.808391 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:07.808496 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:07.808685 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:07.808686 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:07.808688 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0810 07:15:07.808756 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:07.808758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.809462 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.809463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:07.810441 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:07.810444 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.810824 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.810826 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:07.810896 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.810901 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 07:15:07.866940 (+ 56039us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 07:15:07.867141 (+ 201us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.217.126:35945: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":131,"threads_started":1}
I20260810 07:15:07.867751 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 22168
2026-08-10T07:15:07Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (271 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (96 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (95 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:15:08 dist-test-slave-10pc krb5kdc[22285](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:15:08 dist-test-slave-10pc krb5kdc[22285](info): set up 2 sockets
Aug 10 07:15:08 dist-test-slave-10pc krb5kdc[22285](info): commencing operation
krb5kdc: starting...
W20260810 07:15:09.166636 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:15:09 dist-test-slave-10pc krb5kdc[22285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:15:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:15:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
I20260810 07:15:09.297652 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:41281
--webserver_interface=127.15.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:38635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:41281
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:09.384297 22298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:09.384552 22298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:09.384578 22298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:09.385819 22298 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:09.385865 22298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:09.385880 22298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:09.385893 22298 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:09.385905 22298 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:15:09.387354 22298 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:38635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:41281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:41281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:09.387569 22298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:09.387777 22298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:09.392737 22305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:09.392731 22304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:09.392992 22298 server_base.cc:1034] running on GCE node
W20260810 07:15:09.392731 22308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:09.393362 22298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:09.393672 22298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:09.394819 22298 hybrid_clock.cc:648] HybridClock initialized: now 1786346109394803 us; error 32 us; skew 500 ppm
Aug 10 07:15:09 dist-test-slave-10pc krb5kdc[22285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346109, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:09.397574 22298 init.cc:377] Logged in from keytab as kudu/127.15.217.126@KRBTEST.COM (short username kudu)
I20260810 07:15:09.398061 22298 webserver.cc:460] Webserver started at http://127.15.217.126:39585/ using document root <none> and password file <none>
I20260810 07:15:09.398401 22298 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:09.398484 22298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:09.398671 22298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:09.399926 22298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "79d7d9f3039947359d53f0113263b557"
format_stamp: "Formatted at 2026-08-10 07:15:09 on dist-test-slave-10pc"
server_key: "f44c3b062cb89c1dfcce26b29581e022"
server_key_iv: "dfc3abefaf8e37d5f9232cd380fdcf98"
server_key_version: "encryptionkey@0"
I20260810 07:15:09.400334 22298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "79d7d9f3039947359d53f0113263b557"
format_stamp: "Formatted at 2026-08-10 07:15:09 on dist-test-slave-10pc"
server_key: "f44c3b062cb89c1dfcce26b29581e022"
server_key_iv: "dfc3abefaf8e37d5f9232cd380fdcf98"
server_key_version: "encryptionkey@0"
I20260810 07:15:09.401780 22298 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:15:09.402535 22316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:09.402804 22298 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:15:09.403002 22298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "79d7d9f3039947359d53f0113263b557"
format_stamp: "Formatted at 2026-08-10 07:15:09 on dist-test-slave-10pc"
server_key: "f44c3b062cb89c1dfcce26b29581e022"
server_key_iv: "dfc3abefaf8e37d5f9232cd380fdcf98"
server_key_version: "encryptionkey@0"
I20260810 07:15:09.403095 22298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:09.415904 22298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:09.418553 22298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:09.418738 22298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:09.423266 22298 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:41281
I20260810 07:15:09.423306 22371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:41281 every 8 connection(s)
I20260810 07:15:09.423686 22298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:15:09.424672 22372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:09.427067 22372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79d7d9f3039947359d53f0113263b557: Bootstrap starting.
I20260810 07:15:09.427703 22372 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79d7d9f3039947359d53f0113263b557: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:09.428018 22372 log.cc:824] T 00000000000000000000000000000000 P 79d7d9f3039947359d53f0113263b557: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:09.428782 22372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79d7d9f3039947359d53f0113263b557: No bootstrap required, opened a new log
I20260810 07:15:09.430446 22372 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79d7d9f3039947359d53f0113263b557 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79d7d9f3039947359d53f0113263b557" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41281 } }
I20260810 07:15:09.430594 22372 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79d7d9f3039947359d53f0113263b557 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:09.430616 22372 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79d7d9f3039947359d53f0113263b557 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79d7d9f3039947359d53f0113263b557, State: Initialized, Role: FOLLOWER
I20260810 07:15:09.430799 22372 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79d7d9f3039947359d53f0113263b557 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79d7d9f3039947359d53f0113263b557" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41281 } }
I20260810 07:15:09.430863 22372 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79d7d9f3039947359d53f0113263b557 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:15:09.430886 22372 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79d7d9f3039947359d53f0113263b557 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:15:09.430948 22372 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79d7d9f3039947359d53f0113263b557 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:09.431520 22372 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79d7d9f3039947359d53f0113263b557 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79d7d9f3039947359d53f0113263b557" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41281 } }
I20260810 07:15:09.431613 22372 leader_election.cc:302] T 00000000000000000000000000000000 P 79d7d9f3039947359d53f0113263b557 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79d7d9f3039947359d53f0113263b557; no voters:
I20260810 07:15:09.431788 22372 leader_election.cc:288] T 00000000000000000000000000000000 P 79d7d9f3039947359d53f0113263b557 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:15:09.431850 22375 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79d7d9f3039947359d53f0113263b557 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:09.432085 22372 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79d7d9f3039947359d53f0113263b557 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:09.432263 22375 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79d7d9f3039947359d53f0113263b557 [term 1 LEADER]: Becoming Leader. State: Replica: 79d7d9f3039947359d53f0113263b557, State: Running, Role: LEADER
I20260810 07:15:09.432425 22375 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79d7d9f3039947359d53f0113263b557 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79d7d9f3039947359d53f0113263b557" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41281 } }
I20260810 07:15:09.433017 22377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79d7d9f3039947359d53f0113263b557 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79d7d9f3039947359d53f0113263b557. Latest consensus state: current_term: 1 leader_uuid: "79d7d9f3039947359d53f0113263b557" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79d7d9f3039947359d53f0113263b557" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41281 } } }
I20260810 07:15:09.433102 22377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79d7d9f3039947359d53f0113263b557 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:09.433149 22376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79d7d9f3039947359d53f0113263b557 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79d7d9f3039947359d53f0113263b557" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79d7d9f3039947359d53f0113263b557" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41281 } } }
I20260810 07:15:09.433229 22376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79d7d9f3039947359d53f0113263b557 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:09.433598 22385 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:09.433923 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 22298
I20260810 07:15:09.434049 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:15:09.434165 22385 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:15:09.434314 16229 external_mini_cluster.cc:1428] Setting key de66112c0692b637d6e40c98bfabca08
I20260810 07:15:09.435863 22385 catalog_manager.cc:1347] Generated new cluster ID: 1ded96ca1bb1431f926374e8f58f2cf3
I20260810 07:15:09.435910 22385 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 07:15:09 dist-test-slave-10pc krb5kdc[22285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:15:09.451785 22385 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:15:09.452329 22385 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:09.456200 22396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:09.434823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59412 (local address 127.15.217.126:41281)
0810 07:15:09.435011 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:09.435022 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:09.444755 (+ 9733us) server_negotiation.cc:407] Connection header received
0810 07:15:09.444941 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:09.444950 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:09.445013 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:09.445161 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:09.445986 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:09.447508 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:09.448584 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:09.449280 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:09.449401 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:09.452419 (+ 3018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:09.452424 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:09.452424 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:09.452439 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:09.454464 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:09.454900 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:09.454903 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:09.454903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:09.455017 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:09.455263 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:09.455264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:09.455264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:09.455540 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:09.455710 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:09.455808 (+ 98us) server_negotiation.cc:299] Negotiation successful
0810 07:15:09.455956 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":100,"threads_started":1}
I20260810 07:15:09.457468 22385 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79d7d9f3039947359d53f0113263b557: Generated new TSK 0
I20260810 07:15:09.457641 22385 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab.
Entry for principal kudu/127.15.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab.
I20260810 07:15:09.487982 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.65:0
--local_ip_for_outbound_sockets=127.15.217.65
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:41281
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:38635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:09.574483 22400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:09.574753 22400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:09.574793 22400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:09.575975 22400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:09.576049 22400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:09.576095 22400 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.65
I20260810 07:15:09.577667 22400 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:38635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:41281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.65
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:09.577966 22400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:09.578218 22400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:09.583484 22406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:09.583532 22405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:09.583487 22409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:09.583740 22400 server_base.cc:1034] running on GCE node
I20260810 07:15:09.584105 22400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:09.584363 22400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:09.585532 22400 hybrid_clock.cc:648] HybridClock initialized: now 1786346109585535 us; error 47 us; skew 500 ppm
Aug 10 07:15:09 dist-test-slave-10pc krb5kdc[22285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346109, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:09.588274 22400 init.cc:377] Logged in from keytab as kudu/127.15.217.65@KRBTEST.COM (short username kudu)
I20260810 07:15:09.588797 22400 webserver.cc:460] Webserver started at http://127.15.217.65:42063/ using document root <none> and password file <none>
I20260810 07:15:09.589071 22400 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:09.589154 22400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:09.589308 22400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:09.590402 22400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-0/data/instance:
uuid: "8f15de4488624750a518426b537d8e00"
format_stamp: "Formatted at 2026-08-10 07:15:09 on dist-test-slave-10pc"
server_key: "d646884671ae55c75a6d34981fc0d95e"
server_key_iv: "76cf0309d291b05b5b4e5bac6e295b3e"
server_key_version: "encryptionkey@0"
I20260810 07:15:09.590792 22400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance:
uuid: "8f15de4488624750a518426b537d8e00"
format_stamp: "Formatted at 2026-08-10 07:15:09 on dist-test-slave-10pc"
server_key: "d646884671ae55c75a6d34981fc0d95e"
server_key_iv: "76cf0309d291b05b5b4e5bac6e295b3e"
server_key_version: "encryptionkey@0"
I20260810 07:15:09.592252 22400 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:15:09.593081 22416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:09.593271 22400 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:09.593381 22400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-0/wal
uuid: "8f15de4488624750a518426b537d8e00"
format_stamp: "Formatted at 2026-08-10 07:15:09 on dist-test-slave-10pc"
server_key: "d646884671ae55c75a6d34981fc0d95e"
server_key_iv: "76cf0309d291b05b5b4e5bac6e295b3e"
server_key_version: "encryptionkey@0"
I20260810 07:15:09.593458 22400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:09.605326 22400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:09.607831 22400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:09.608011 22400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:09.608268 22400 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:09.608692 22400 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:09.608757 22400 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:09.608801 22400 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:09.608830 22400 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:09.615190 22400 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.65:38353
I20260810 07:15:09.615231 22537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.65:38353 every 8 connection(s)
I20260810 07:15:09.615568 22400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
Aug 10 07:15:09 dist-test-slave-10pc krb5kdc[22285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346109, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:15:09.624563 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 22400
I20260810 07:15:09.624701 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance
I20260810 07:15:09.624913 16229 external_mini_cluster.cc:1428] Setting key fc6ca26c5b847fed70471eb235eaf374
I20260810 07:15:09.625592 22396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:09.616519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:55149 (local address 127.15.217.126:41281)
0810 07:15:09.616583 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:09.616585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:09.616820 (+ 235us) server_negotiation.cc:407] Connection header received
0810 07:15:09.617811 (+ 991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:09.617812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:09.617822 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:09.617882 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:09.618495 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:09.619000 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:09.620000 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:09.620201 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:09.620252 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:09.622461 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:09.622463 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:09.622463 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:09.622471 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:09.623936 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:09.624361 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:09.624362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:09.624362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:09.624437 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:09.624777 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:09.624779 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:09.624779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:09.624920 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:09.625019 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:09.625388 (+ 369us) server_negotiation.cc:299] Negotiation successful
0810 07:15:09.625500 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 07:15:09.625788 22540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:09.616456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41281 (local address 127.15.217.65:55149)
0810 07:15:09.616672 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:09.616719 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:09.617717 (+ 998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:09.617941 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:09.617944 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:09.618213 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:09.618418 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:09.618422 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:09.619090 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:09.619091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:09.619923 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:09.619925 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:09.620277 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:09.620278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:09.620384 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:09.620929 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:09.620939 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:09.622358 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:09.624009 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:09.624015 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:09.624017 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:09.624249 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:09.624561 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:09.624561 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:09.624562 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:09.624673 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:09.625024 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:09.625032 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:09.625217 (+ 185us) client_negotiation.cc:769] Sending connection context
0810 07:15:09.625344 (+ 127us) client_negotiation.cc:240] Negotiation successful
0810 07:15:09.625515 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":115,"threads_started":1}
I20260810 07:15:09.626278 22538 heartbeater.cc:347] Connected to a master server at 127.15.217.126:41281
I20260810 07:15:09.626380 22538 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:09.626596 22538 heartbeater.cc:511] Master 127.15.217.126:41281 requested a full tablet report, sending...
I20260810 07:15:09.627173 22333 ts_manager.cc:194] Registered new tserver with Master: 8f15de4488624750a518426b537d8e00 (127.15.217.65:38353)
I20260810 07:15:09.628110 22333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.65@KRBTEST.COM'} at 127.15.217.65:55149
WARNING: no policy specified for kudu/127.15.217.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab.
Entry for principal kudu/127.15.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab.
I20260810 07:15:09.651587 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.66:0
--local_ip_for_outbound_sockets=127.15.217.66
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:41281
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:38635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:09.742615 22543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:09.742887 22543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:09.742931 22543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:09.744169 22543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:09.744247 22543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:09.744292 22543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.66
I20260810 07:15:09.745926 22543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:38635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:41281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.66
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:09.746235 22543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:09.746502 22543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:09.751997 22549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:09.752018 22548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:09.751997 22552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:09.752300 22543 server_base.cc:1034] running on GCE node
I20260810 07:15:09.752586 22543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:09.752856 22543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:09.754029 22543 hybrid_clock.cc:648] HybridClock initialized: now 1786346109754023 us; error 38 us; skew 500 ppm
Aug 10 07:15:09 dist-test-slave-10pc krb5kdc[22285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346109, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:09.756837 22543 init.cc:377] Logged in from keytab as kudu/127.15.217.66@KRBTEST.COM (short username kudu)
I20260810 07:15:09.757294 22543 webserver.cc:460] Webserver started at http://127.15.217.66:39455/ using document root <none> and password file <none>
I20260810 07:15:09.757537 22543 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:09.757582 22543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:09.757687 22543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:09.758755 22543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-1/data/instance:
uuid: "e33e18e6f371458893c4c19d6abebbe6"
format_stamp: "Formatted at 2026-08-10 07:15:09 on dist-test-slave-10pc"
server_key: "c86a435f523efdc6118da8b19b55eda0"
server_key_iv: "f21bd05f6ef764c7818c41a3bef84733"
server_key_version: "encryptionkey@0"
I20260810 07:15:09.759073 22543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance:
uuid: "e33e18e6f371458893c4c19d6abebbe6"
format_stamp: "Formatted at 2026-08-10 07:15:09 on dist-test-slave-10pc"
server_key: "c86a435f523efdc6118da8b19b55eda0"
server_key_iv: "f21bd05f6ef764c7818c41a3bef84733"
server_key_version: "encryptionkey@0"
I20260810 07:15:09.760505 22543 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:15:09.761300 22560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:09.761503 22543 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:09.761597 22543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-1/wal
uuid: "e33e18e6f371458893c4c19d6abebbe6"
format_stamp: "Formatted at 2026-08-10 07:15:09 on dist-test-slave-10pc"
server_key: "c86a435f523efdc6118da8b19b55eda0"
server_key_iv: "f21bd05f6ef764c7818c41a3bef84733"
server_key_version: "encryptionkey@0"
I20260810 07:15:09.761659 22543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:09.782590 22543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:09.785218 22543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:09.785346 22543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:09.785543 22543 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:09.785893 22543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:09.785934 22543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:09.785980 22543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:09.786007 22543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:09.792085 22543 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.66:46851
I20260810 07:15:09.792172 22687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.66:46851 every 8 connection(s)
I20260810 07:15:09.792543 22543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
Aug 10 07:15:09 dist-test-slave-10pc krb5kdc[22285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346109, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:15:09.800590 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 22543
I20260810 07:15:09.800726 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance
I20260810 07:15:09.800959 16229 external_mini_cluster.cc:1428] Setting key e24069757814d7ec3ba7829bb17fc78a
I20260810 07:15:09.804559 22396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:09.793348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:42235 (local address 127.15.217.126:41281)
0810 07:15:09.793417 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:09.793421 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:09.793654 (+ 233us) server_negotiation.cc:407] Connection header received
0810 07:15:09.794637 (+ 983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:09.794638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:09.794649 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:09.794708 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:09.795351 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:09.795848 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:09.797024 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:09.797245 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:09.797301 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:09.799565 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:09.799567 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:09.799567 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:09.799575 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:09.801103 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:09.801658 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:09.801659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:09.801660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:09.801735 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:09.803793 (+ 2058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:09.803794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:09.803794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:09.803966 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:09.804046 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:09.804326 (+ 280us) server_negotiation.cc:299] Negotiation successful
0810 07:15:09.804429 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 07:15:09.804780 22690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:09.793333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41281 (local address 127.15.217.66:42235)
0810 07:15:09.793526 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:09.793565 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:09.794533 (+ 968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:09.794766 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:09.794769 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:09.795038 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:09.795272 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:09.795277 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:09.795934 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:09.795934 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:09.796936 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:09.796939 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:09.797313 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:09.797314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:09.797467 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:09.798042 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:09.798051 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:09.799472 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:09.801237 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:09.801246 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:09.801248 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:09.801571 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:09.803584 (+ 2013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:09.803585 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:09.803586 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:09.803713 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:09.804053 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:09.804058 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:09.804187 (+ 129us) client_negotiation.cc:769] Sending connection context
0810 07:15:09.804276 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 07:15:09.804411 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":102,"threads_started":1}
I20260810 07:15:09.805313 22688 heartbeater.cc:347] Connected to a master server at 127.15.217.126:41281
I20260810 07:15:09.805437 22688 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:09.805634 22688 heartbeater.cc:511] Master 127.15.217.126:41281 requested a full tablet report, sending...
I20260810 07:15:09.806123 22333 ts_manager.cc:194] Registered new tserver with Master: e33e18e6f371458893c4c19d6abebbe6 (127.15.217.66:46851)
I20260810 07:15:09.806515 22333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.66@KRBTEST.COM'} at 127.15.217.66:42235
WARNING: no policy specified for kudu/127.15.217.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab.
Entry for principal kudu/127.15.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab.
I20260810 07:15:09.829348 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.67:0
--local_ip_for_outbound_sockets=127.15.217.67
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:41281
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:38635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:09.916991 22693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:09.917227 22693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:09.917254 22693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:09.918484 22693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:09.918534 22693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:09.918562 22693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.67
I20260810 07:15:09.920116 22693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:38635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:41281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.67
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:09.920387 22693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:09.920678 22693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:09.925853 22700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:09.925853 22703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:09.925906 22701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:09.926347 22693 server_base.cc:1034] running on GCE node
I20260810 07:15:09.926539 22693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:09.926746 22693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:09.927892 22693 hybrid_clock.cc:648] HybridClock initialized: now 1786346109927875 us; error 30 us; skew 500 ppm
Aug 10 07:15:09 dist-test-slave-10pc krb5kdc[22285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346109, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:09.930527 22693 init.cc:377] Logged in from keytab as kudu/127.15.217.67@KRBTEST.COM (short username kudu)
I20260810 07:15:09.930935 22693 webserver.cc:460] Webserver started at http://127.15.217.67:39497/ using document root <none> and password file <none>
I20260810 07:15:09.931150 22693 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:09.931200 22693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:09.931295 22693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:09.932407 22693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-2/data/instance:
uuid: "92f36bb033634a099c636a6898dfd2ae"
format_stamp: "Formatted at 2026-08-10 07:15:09 on dist-test-slave-10pc"
server_key: "23fba440025b6a68170759602f218d4c"
server_key_iv: "a055d7ee29fe96c4801d5a42f1b765c7"
server_key_version: "encryptionkey@0"
I20260810 07:15:09.932826 22693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance:
uuid: "92f36bb033634a099c636a6898dfd2ae"
format_stamp: "Formatted at 2026-08-10 07:15:09 on dist-test-slave-10pc"
server_key: "23fba440025b6a68170759602f218d4c"
server_key_iv: "a055d7ee29fe96c4801d5a42f1b765c7"
server_key_version: "encryptionkey@0"
I20260810 07:15:09.934434 22693 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260810 07:15:09.935133 22711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:09.935322 22693 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:09.935401 22693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-2/wal
uuid: "92f36bb033634a099c636a6898dfd2ae"
format_stamp: "Formatted at 2026-08-10 07:15:09 on dist-test-slave-10pc"
server_key: "23fba440025b6a68170759602f218d4c"
server_key_iv: "a055d7ee29fe96c4801d5a42f1b765c7"
server_key_version: "encryptionkey@0"
I20260810 07:15:09.935459 22693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:09.945441 22693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:09.947283 22693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:09.947400 22693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:09.947593 22693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:09.947853 22693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:09.947886 22693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:09.947909 22693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:09.947922 22693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:09.954067 22693 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.67:45353
I20260810 07:15:09.954114 22835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.67:45353 every 8 connection(s)
I20260810 07:15:09.954454 22693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
I20260810 07:15:09.955566 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 22693
I20260810 07:15:09.955658 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance
I20260810 07:15:09.955891 16229 external_mini_cluster.cc:1428] Setting key 09d18e6a287140423d2d734a050ba766
Aug 10 07:15:09 dist-test-slave-10pc krb5kdc[22285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346109, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:15:09.963969 22396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:09.955220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:50077 (local address 127.15.217.126:41281)
0810 07:15:09.955294 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:09.955296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:09.955523 (+ 227us) server_negotiation.cc:407] Connection header received
0810 07:15:09.956856 (+ 1333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:09.956857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:09.956867 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:09.956931 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:09.957614 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:09.958183 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:09.959152 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:09.959344 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:09.959395 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:09.961557 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:09.961558 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:09.961559 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:09.961566 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:09.962784 (+ 1218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:09.963155 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:09.963156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:09.963156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:09.963220 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:09.963434 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:09.963435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:09.963435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:09.963538 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:09.963588 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:09.963829 (+ 241us) server_negotiation.cc:299] Negotiation successful
0810 07:15:09.963899 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:15:09.964192 22838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:09.955210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41281 (local address 127.15.217.67:50077)
0810 07:15:09.955395 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:09.955439 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:09.956742 (+ 1303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:09.956989 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:09.956993 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:09.957288 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:09.957493 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:09.957533 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:09.958268 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:09.958269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:09.959075 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:09.959077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:09.959417 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:09.959418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:09.959522 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:09.960078 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:09.960086 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:09.961471 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:09.962862 (+ 1391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:09.962868 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:09.962870 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:09.963089 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:09.963279 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:09.963280 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:09.963280 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:09.963380 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:09.963592 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:09.963596 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:09.963714 (+ 118us) client_negotiation.cc:769] Sending connection context
0810 07:15:09.963797 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 07:15:09.963924 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":111,"threads_started":1}
I20260810 07:15:09.964730 22836 heartbeater.cc:347] Connected to a master server at 127.15.217.126:41281
I20260810 07:15:09.964828 22836 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:09.965006 22836 heartbeater.cc:511] Master 127.15.217.126:41281 requested a full tablet report, sending...
I20260810 07:15:09.965422 22333 ts_manager.cc:194] Registered new tserver with Master: 92f36bb033634a099c636a6898dfd2ae (127.15.217.67:45353)
I20260810 07:15:09.965837 22333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.67@KRBTEST.COM'} at 127.15.217.67:50077
I20260810 07:15:09.967985 16229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/krb5kdc/test-admin.keytab.
Aug 10 07:15:09 dist-test-slave-10pc krb5kdc[22285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:09.985683 16229 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260810 07:15:09.987381 22843 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:09.986600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37962 (local address 127.15.217.65:38353)
0810 07:15:09.986776 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:09.986786 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:09.986797 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:15:09.986829 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:09.986831 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:09.986845 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:09.986924 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:09.987266 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37962: BlockingRecv error: recv got EOF from 127.0.0.1:37962 (error 108)
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":96,"threads_started":1}
W20260810 07:15:09.987535 16229 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/krb5kdc/krb5cc' of type FILE
Aug 10 07:15:09 dist-test-slave-10pc krb5kdc[22285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:09.988669 16229 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 07:15:09 dist-test-slave-10pc krb5kdc[22285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:09.989661 16229 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 07:15:09 dist-test-slave-10pc krb5kdc[22285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.65@KRBTEST.COM
I20260810 07:15:09.998188 22843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:09.990274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37964 (local address 127.15.217.65:38353)
0810 07:15:09.990336 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:09.990338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:09.990388 (+ 50us) server_negotiation.cc:407] Connection header received
0810 07:15:09.990441 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:09.990442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:09.990453 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:09.990504 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:09.990902 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:09.991661 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:09.992377 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:09.992795 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:09.992856 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:09.994986 (+ 2130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:09.994997 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:09.994998 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:09.995005 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:09.996891 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:09.997287 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:09.997292 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:09.997292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:09.997362 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:09.997587 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:09.997588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:09.997589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:09.997755 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:09.997921 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:09.997976 (+ 55us) server_negotiation.cc:299] Negotiation successful
0810 07:15:09.998065 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260810 07:15:10.000233 22843 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:09.999562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37980 (local address 127.15.217.65:38353)
0810 07:15:09.999612 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:09.999614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:09.999672 (+ 58us) server_negotiation.cc:407] Connection header received
0810 07:15:09.999712 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:09.999712 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:09.999722 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:09.999772 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:10.000168 (+ 396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37980: BlockingRecv error: recv got EOF from 127.0.0.1:37980 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260810 07:15:10.000363 16229 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/krb5kdc/krb5cc' of type FILE
Aug 10 07:15:09 dist-test-slave-10pc krb5kdc[22285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:10.001453 16229 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 07:15:09 dist-test-slave-10pc krb5kdc[22285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:10.002338 16229 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 07:15:10 dist-test-slave-10pc krb5kdc[22285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.65@KRBTEST.COM
I20260810 07:15:10.010162 22843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:10.003026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37994 (local address 127.15.217.65:38353)
0810 07:15:10.003092 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:10.003094 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:10.003104 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:15:10.003130 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:10.003131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:10.003148 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:10.003211 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:10.003586 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:10.004089 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:10.004901 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:10.005117 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:10.005198 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:10.007410 (+ 2212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:10.007411 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:10.007412 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:10.007419 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:10.008798 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:10.009134 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:10.009134 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:10.009135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:10.009196 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:10.009412 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:10.009414 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:10.009414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:10.009564 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:10.009666 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:10.009834 (+ 168us) server_negotiation.cc:299] Negotiation successful
0810 07:15:10.010046 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260810 07:15:10.012007 16229 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346081759552-16229-0/krb5kdc/krb5cc' of type FILE
W20260810 07:15:10.012004 22843 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:10.011366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38010 (local address 127.15.217.65:38353)
0810 07:15:10.011407 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:10.011409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:10.011441 (+ 32us) server_negotiation.cc:407] Connection header received
0810 07:15:10.011496 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:10.011496 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:10.011504 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:10.011555 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:10.011929 (+ 374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38010: BlockingRecv error: recv got EOF from 127.0.0.1:38010 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 10 07:15:10 dist-test-slave-10pc krb5kdc[22285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:10.013229 16229 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 07:15:10 dist-test-slave-10pc krb5kdc[22285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:10.014192 16229 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 07:15:10 dist-test-slave-10pc krb5kdc[22285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.65@KRBTEST.COM
I20260810 07:15:10.021631 22843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:10.014888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38022 (local address 127.15.217.65:38353)
0810 07:15:10.014958 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:10.014961 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:10.014971 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:15:10.015001 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:10.015003 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:10.015015 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:10.015079 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:10.015471 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:10.016038 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:10.016941 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:10.017152 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:10.017217 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:10.019337 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:10.019338 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:10.019338 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:10.019345 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:10.020702 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:10.021032 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:10.021032 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:10.021033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:10.021084 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:10.021252 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:10.021253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:10.021253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:10.021346 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:10.021422 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:10.021502 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 07:15:10.021563 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:15:10.022348 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 22400
I20260810 07:15:10.033394 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 22543
I20260810 07:15:10.043885 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 22693
I20260810 07:15:10.050642 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 22298
2026-08-10T07:15:10Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1991 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:15:10 dist-test-slave-10pc krb5kdc[22857](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:15:10 dist-test-slave-10pc krb5kdc[22857](info): set up 2 sockets
Aug 10 07:15:10 dist-test-slave-10pc krb5kdc[22857](info): commencing operation
krb5kdc: starting...
W20260810 07:15:11.143728 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:15:11 dist-test-slave-10pc krb5kdc[22857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346111, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:15:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:15:11Z Disabled control of system clock
WARNING: no policy specified for oryx/127.15.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.126.keytab.
Entry for principal oryx/127.15.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.126.keytab.
I20260810 07:15:11.279959 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:41187
--webserver_interface=127.15.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.126.keytab
--principal=oryx/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:44299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:41187
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:11.367429 22871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:11.367705 22871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:11.367748 22871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:11.368965 22871 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:11.369040 22871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:11.369065 22871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:11.369086 22871 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:11.369117 22871 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:15:11.370692 22871 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:44299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:41187
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.126.keytab
--principal=oryx/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:41187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:11.370998 22871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:11.371259 22871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:11.376468 22877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:11.376441 22879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:11.376520 22876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:11.376921 22871 server_base.cc:1034] running on GCE node
I20260810 07:15:11.377368 22871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:11.377717 22871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:11.378897 22871 hybrid_clock.cc:648] HybridClock initialized: now 1786346111378866 us; error 53 us; skew 500 ppm
Aug 10 07:15:11 dist-test-slave-10pc krb5kdc[22857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346111, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:11.381771 22871 init.cc:377] Logged in from keytab as oryx/127.15.217.126@KRBTEST.COM (short username oryx)
I20260810 07:15:11.382253 22871 webserver.cc:460] Webserver started at http://127.15.217.126:33873/ using document root <none> and password file <none>
I20260810 07:15:11.382526 22871 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:11.382601 22871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:11.382758 22871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:11.383882 22871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "29e10cc613d84f3e807c6339559f7930"
format_stamp: "Formatted at 2026-08-10 07:15:11 on dist-test-slave-10pc"
server_key: "9cfb5ce32f8677399692007538931eb7"
server_key_iv: "8a778e3281109b4d21af78ad1b03e876"
server_key_version: "encryptionkey@0"
I20260810 07:15:11.384275 22871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "29e10cc613d84f3e807c6339559f7930"
format_stamp: "Formatted at 2026-08-10 07:15:11 on dist-test-slave-10pc"
server_key: "9cfb5ce32f8677399692007538931eb7"
server_key_iv: "8a778e3281109b4d21af78ad1b03e876"
server_key_version: "encryptionkey@0"
I20260810 07:15:11.385794 22871 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:15:11.386582 22889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:11.386766 22871 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:15:11.386881 22871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "29e10cc613d84f3e807c6339559f7930"
format_stamp: "Formatted at 2026-08-10 07:15:11 on dist-test-slave-10pc"
server_key: "9cfb5ce32f8677399692007538931eb7"
server_key_iv: "8a778e3281109b4d21af78ad1b03e876"
server_key_version: "encryptionkey@0"
I20260810 07:15:11.386965 22871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:11.403515 22871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:11.406221 22871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:11.406402 22871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:11.410818 22871 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:41187
I20260810 07:15:11.410861 22952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:41187 every 8 connection(s)
I20260810 07:15:11.411275 22871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:15:11.412411 22953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:11.415004 22953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930: Bootstrap starting.
I20260810 07:15:11.415769 22953 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:11.416172 22953 log.cc:824] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:11.416639 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 22871
I20260810 07:15:11.416759 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:15:11.416990 16229 external_mini_cluster.cc:1428] Setting key b6d176c905ac5d13bcb82a5f12b9349d
I20260810 07:15:11.417256 22953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930: No bootstrap required, opened a new log
I20260810 07:15:11.419916 22953 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } }
I20260810 07:15:11.420091 22953 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:11.420140 22953 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29e10cc613d84f3e807c6339559f7930, State: Initialized, Role: FOLLOWER
I20260810 07:15:11.420305 22953 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } }
I20260810 07:15:11.420403 22953 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:15:11.420467 22953 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:15:11.420524 22953 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:11.421137 22953 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } }
I20260810 07:15:11.421262 22953 leader_election.cc:302] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29e10cc613d84f3e807c6339559f7930; no voters:
I20260810 07:15:11.421468 22953 leader_election.cc:288] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:15:11.421490 22959 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:11.421669 22959 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [term 1 LEADER]: Becoming Leader. State: Replica: 29e10cc613d84f3e807c6339559f7930, State: Running, Role: LEADER
I20260810 07:15:11.421777 22953 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:11.421816 22959 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } }
I20260810 07:15:11.423032 22960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "29e10cc613d84f3e807c6339559f7930" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } }
I20260810 07:15:11.423135 22960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:11.423439 22961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29e10cc613d84f3e807c6339559f7930. Latest consensus state: current_term: 1 leader_uuid: "29e10cc613d84f3e807c6339559f7930" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } }
I20260810 07:15:11.423822 22961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [sys.catalog]: This master's current role is: LEADER
W20260810 07:15:11.424125 22975 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:15:11.424183 22975 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:15:11.424239 22976 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:11.424427 22976 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 07:15:11 dist-test-slave-10pc krb5kdc[22857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346111, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM
I20260810 07:15:11.426126 22976 catalog_manager.cc:1347] Generated new cluster ID: 213dbfc3d5314eb3b1002679033b6e06
I20260810 07:15:11.426185 22976 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:15:11.429672 22957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:11.417637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51156 (local address 127.15.217.126:41187)
0810 07:15:11.417849 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:11.417860 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:11.417897 (+ 37us) server_negotiation.cc:407] Connection header received
0810 07:15:11.418835 (+ 938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:11.418844 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:11.418901 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:11.419024 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:11.419841 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:11.421178 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:11.422192 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:11.422891 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:11.423018 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:11.425936 (+ 2918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:11.425940 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:11.425941 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:11.425954 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:11.427891 (+ 1937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:11.428308 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:11.428312 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:11.428312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:11.428411 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:11.428683 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:11.428684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:11.428685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:11.428947 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:11.429128 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:11.429212 (+ 84us) server_negotiation.cc:299] Negotiation successful
0810 07:15:11.429470 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":159,"thread_start_us":119,"threads_started":1}
I20260810 07:15:11.440764 22976 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:15:11.441340 22976 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:11.451076 22976 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930: Generated new TSK 0
I20260810 07:15:11.451328 22976 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.15.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.65.keytab.
Entry for principal oryx/127.15.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.65.keytab.
I20260810 07:15:11.489022 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.65:0
--local_ip_for_outbound_sockets=127.15.217.65
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:41187
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.65.keytab
--principal=oryx/127.15.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:44299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:11.579447 22981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:11.579658 22981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:11.579691 22981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:11.581018 22981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:11.581065 22981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:11.581094 22981 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.65
I20260810 07:15:11.582763 22981 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:44299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.65.keytab
--principal=oryx/127.15.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:41187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.65
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:11.583060 22981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:11.583331 22981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:11.589015 22989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:11.589097 22988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:11.589012 22992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:11.589294 22981 server_base.cc:1034] running on GCE node
I20260810 07:15:11.589529 22981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:11.589839 22981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:11.591012 22981 hybrid_clock.cc:648] HybridClock initialized: now 1786346111591007 us; error 40 us; skew 500 ppm
Aug 10 07:15:11 dist-test-slave-10pc krb5kdc[22857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346111, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:11.593767 22981 init.cc:377] Logged in from keytab as oryx/127.15.217.65@KRBTEST.COM (short username oryx)
I20260810 07:15:11.594259 22981 webserver.cc:460] Webserver started at http://127.15.217.65:41719/ using document root <none> and password file <none>
I20260810 07:15:11.594532 22981 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:11.594580 22981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:11.594722 22981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:11.595861 22981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-0/data/instance:
uuid: "2a7624fba8924223967850a43cc913c2"
format_stamp: "Formatted at 2026-08-10 07:15:11 on dist-test-slave-10pc"
server_key: "a2f3ec1c5625a27ee5b497ceb110f772"
server_key_iv: "9b13edfdbe62fc386dde93af3a86c603"
server_key_version: "encryptionkey@0"
I20260810 07:15:11.596247 22981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance:
uuid: "2a7624fba8924223967850a43cc913c2"
format_stamp: "Formatted at 2026-08-10 07:15:11 on dist-test-slave-10pc"
server_key: "a2f3ec1c5625a27ee5b497ceb110f772"
server_key_iv: "9b13edfdbe62fc386dde93af3a86c603"
server_key_version: "encryptionkey@0"
I20260810 07:15:11.597771 22981 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:11.598628 23000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:11.598834 22981 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:11.598951 22981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-0/wal
uuid: "2a7624fba8924223967850a43cc913c2"
format_stamp: "Formatted at 2026-08-10 07:15:11 on dist-test-slave-10pc"
server_key: "a2f3ec1c5625a27ee5b497ceb110f772"
server_key_iv: "9b13edfdbe62fc386dde93af3a86c603"
server_key_version: "encryptionkey@0"
I20260810 07:15:11.599032 22981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:11.615343 22981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:11.617694 22981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:11.617883 22981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:11.618146 22981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:11.618496 22981 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:11.618554 22981 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:11.618589 22981 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:11.618633 22981 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:11.625598 22981 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.65:34201
I20260810 07:15:11.625723 23122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.65:34201 every 8 connection(s)
I20260810 07:15:11.626051 22981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
Aug 10 07:15:11 dist-test-slave-10pc krb5kdc[22857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346111, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM
I20260810 07:15:11.635880 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 22981
I20260810 07:15:11.636015 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance
I20260810 07:15:11.636231 16229 external_mini_cluster.cc:1428] Setting key 88d9c6367c0f8854cf9ebde49b3add58
I20260810 07:15:11.637135 22957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:11.627112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:34245 (local address 127.15.217.126:41187)
0810 07:15:11.627185 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:11.627187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:11.627448 (+ 261us) server_negotiation.cc:407] Connection header received
0810 07:15:11.628091 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:11.628094 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:11.628109 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:11.628176 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:11.629404 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:11.630069 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:11.631336 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:11.631579 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:11.631672 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:11.633945 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:11.633947 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:11.633947 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:11.633956 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:11.635295 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:11.635806 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:11.635808 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:11.635808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:11.635879 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:11.636236 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:11.636238 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:11.636238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:11.636428 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:11.636567 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:11.636916 (+ 349us) server_negotiation.cc:299] Negotiation successful
0810 07:15:11.637026 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 07:15:11.637301 23125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:11.627093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41187 (local address 127.15.217.65:34245)
0810 07:15:11.627288 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:11.627336 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:11.627957 (+ 621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:11.628261 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:11.628265 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:11.629050 (+ 785us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:11.629279 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:11.629286 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:11.630328 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:11.630330 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:11.631228 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:11.631231 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:11.631688 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:11.631690 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:11.631825 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:11.632327 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:11.632336 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:11.633832 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:11.635419 (+ 1587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:11.635427 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:11.635430 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:11.635716 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:11.635965 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:11.635966 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:11.635967 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:11.636148 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:11.636561 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:11.636584 (+ 23us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:11.636764 (+ 180us) client_negotiation.cc:769] Sending connection context
0810 07:15:11.636864 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 07:15:11.637068 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":89,"threads_started":1}
I20260810 07:15:11.637835 23123 heartbeater.cc:347] Connected to a master server at 127.15.217.126:41187
I20260810 07:15:11.637955 23123 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:11.638186 23123 heartbeater.cc:511] Master 127.15.217.126:41187 requested a full tablet report, sending...
I20260810 07:15:11.638754 22908 ts_manager.cc:194] Registered new tserver with Master: 2a7624fba8924223967850a43cc913c2 (127.15.217.65:34201)
I20260810 07:15:11.639384 22908 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.217.65@KRBTEST.COM'} at 127.15.217.65:34245
WARNING: no policy specified for oryx/127.15.217.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.217.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.66.keytab.
Entry for principal oryx/127.15.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.66.keytab.
I20260810 07:15:11.666913 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.66:0
--local_ip_for_outbound_sockets=127.15.217.66
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:41187
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.66.keytab
--principal=oryx/127.15.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:44299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:11.753872 23128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:11.754096 23128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:11.754128 23128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:11.755370 23128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:11.755419 23128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:11.755476 23128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.66
I20260810 07:15:11.757006 23128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:44299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.66.keytab
--principal=oryx/127.15.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:41187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.66
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:11.757275 23128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:11.757535 23128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:11.762809 23136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:11.762817 23133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:11.762850 23128 server_base.cc:1034] running on GCE node
W20260810 07:15:11.762832 23134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:11.763206 23128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:11.763470 23128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:11.764631 23128 hybrid_clock.cc:648] HybridClock initialized: now 1786346111764611 us; error 46 us; skew 500 ppm
Aug 10 07:15:11 dist-test-slave-10pc krb5kdc[22857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346111, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:11.767299 23128 init.cc:377] Logged in from keytab as oryx/127.15.217.66@KRBTEST.COM (short username oryx)
I20260810 07:15:11.767709 23128 webserver.cc:460] Webserver started at http://127.15.217.66:35605/ using document root <none> and password file <none>
I20260810 07:15:11.767899 23128 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:11.767941 23128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:11.768030 23128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:11.769203 23128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-1/data/instance:
uuid: "87f6de6d8282454f9da57f1a17b89e6e"
format_stamp: "Formatted at 2026-08-10 07:15:11 on dist-test-slave-10pc"
server_key: "4877ab08011f1be5808742ebd46b4ac8"
server_key_iv: "79fc02607bba5559ccc79e6388bdcc25"
server_key_version: "encryptionkey@0"
I20260810 07:15:11.769541 23128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance:
uuid: "87f6de6d8282454f9da57f1a17b89e6e"
format_stamp: "Formatted at 2026-08-10 07:15:11 on dist-test-slave-10pc"
server_key: "4877ab08011f1be5808742ebd46b4ac8"
server_key_iv: "79fc02607bba5559ccc79e6388bdcc25"
server_key_version: "encryptionkey@0"
I20260810 07:15:11.770869 23128 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:15:11.771588 23145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:11.771757 23128 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 07:15:11.771847 23128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-1/wal
uuid: "87f6de6d8282454f9da57f1a17b89e6e"
format_stamp: "Formatted at 2026-08-10 07:15:11 on dist-test-slave-10pc"
server_key: "4877ab08011f1be5808742ebd46b4ac8"
server_key_iv: "79fc02607bba5559ccc79e6388bdcc25"
server_key_version: "encryptionkey@0"
I20260810 07:15:11.771907 23128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:11.799799 23128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:11.801954 23128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:11.802080 23128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:11.802404 23128 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:11.802765 23128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:11.802801 23128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:11.802824 23128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:11.802870 23128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:11.808954 23128 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.66:46239
I20260810 07:15:11.809049 23265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.66:46239 every 8 connection(s)
I20260810 07:15:11.809377 23128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
I20260810 07:15:11.813508 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 23128
I20260810 07:15:11.813623 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance
I20260810 07:15:11.813824 16229 external_mini_cluster.cc:1428] Setting key 625d81222b3531cfaaad68c1fe4160e2
Aug 10 07:15:11 dist-test-slave-10pc krb5kdc[22857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346111, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM
I20260810 07:15:11.820075 22957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:11.810163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:53469 (local address 127.15.217.126:41187)
0810 07:15:11.810234 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:11.810236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:11.810466 (+ 230us) server_negotiation.cc:407] Connection header received
0810 07:15:11.811000 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:11.811001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:11.811012 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:11.811067 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:11.812070 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:11.812660 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:11.814023 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:11.814298 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:11.814356 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:11.816930 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:11.816932 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:11.816932 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:11.816941 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:11.818592 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:11.819026 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:11.819027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:11.819027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:11.819100 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:11.819352 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:11.819353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:11.819354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:11.819532 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:11.819612 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:11.819890 (+ 278us) server_negotiation.cc:299] Negotiation successful
0810 07:15:11.819980 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 07:15:11.820281 23268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:11.810151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41187 (local address 127.15.217.66:53469)
0810 07:15:11.810356 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:11.810395 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:11.810912 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:11.811126 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:11.811130 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:11.811770 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:11.811987 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:11.811992 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:11.812749 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:11.812750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:11.813851 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:11.813855 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:11.814382 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:11.814383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:11.814532 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:11.815098 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:11.815110 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:11.816823 (+ 1713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:11.818691 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:11.818700 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:11.818703 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:11.818944 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:11.819172 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:11.819173 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:11.819173 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:11.819279 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:11.819615 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:11.819620 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:11.819741 (+ 121us) client_negotiation.cc:769] Sending connection context
0810 07:15:11.819858 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 07:15:11.820032 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":107,"threads_started":1}
I20260810 07:15:11.820868 23266 heartbeater.cc:347] Connected to a master server at 127.15.217.126:41187
I20260810 07:15:11.820996 23266 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:11.821244 23266 heartbeater.cc:511] Master 127.15.217.126:41187 requested a full tablet report, sending...
I20260810 07:15:11.821692 22908 ts_manager.cc:194] Registered new tserver with Master: 87f6de6d8282454f9da57f1a17b89e6e (127.15.217.66:46239)
I20260810 07:15:11.822300 22908 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.217.66@KRBTEST.COM'} at 127.15.217.66:53469
WARNING: no policy specified for oryx/127.15.217.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.217.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.67.keytab.
Entry for principal oryx/127.15.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.67.keytab.
I20260810 07:15:11.842692 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.67:0
--local_ip_for_outbound_sockets=127.15.217.67
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:41187
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.67.keytab
--principal=oryx/127.15.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:44299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:11.930881 23271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:11.931097 23271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:11.931121 23271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:11.932232 23271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:11.932267 23271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:11.932293 23271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.67
I20260810 07:15:11.933799 23271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:44299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.67.keytab
--principal=oryx/127.15.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:41187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.67
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:11.933996 23271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:11.934181 23271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:11.939353 23277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:11.939374 23279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:11.939394 23276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:11.939662 23271 server_base.cc:1034] running on GCE node
I20260810 07:15:11.939886 23271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:11.940151 23271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:11.941349 23271 hybrid_clock.cc:648] HybridClock initialized: now 1786346111941334 us; error 52 us; skew 500 ppm
Aug 10 07:15:11 dist-test-slave-10pc krb5kdc[22857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346111, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:11.944089 23271 init.cc:377] Logged in from keytab as oryx/127.15.217.67@KRBTEST.COM (short username oryx)
I20260810 07:15:11.944643 23271 webserver.cc:460] Webserver started at http://127.15.217.67:44561/ using document root <none> and password file <none>
I20260810 07:15:11.944897 23271 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:11.944990 23271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:11.945147 23271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:11.946271 23271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-2/data/instance:
uuid: "ceed6041617a45b6ab60d5fb2064d12d"
format_stamp: "Formatted at 2026-08-10 07:15:11 on dist-test-slave-10pc"
server_key: "ce68b1ef4a6d2af284f63425f5748de5"
server_key_iv: "0a1ee665952f346aee60aa8a4a08288f"
server_key_version: "encryptionkey@0"
I20260810 07:15:11.946663 23271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance:
uuid: "ceed6041617a45b6ab60d5fb2064d12d"
format_stamp: "Formatted at 2026-08-10 07:15:11 on dist-test-slave-10pc"
server_key: "ce68b1ef4a6d2af284f63425f5748de5"
server_key_iv: "0a1ee665952f346aee60aa8a4a08288f"
server_key_version: "encryptionkey@0"
I20260810 07:15:11.948069 23271 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:15:11.948894 23287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:11.949074 23271 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:11.949175 23271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-2/wal
uuid: "ceed6041617a45b6ab60d5fb2064d12d"
format_stamp: "Formatted at 2026-08-10 07:15:11 on dist-test-slave-10pc"
server_key: "ce68b1ef4a6d2af284f63425f5748de5"
server_key_iv: "0a1ee665952f346aee60aa8a4a08288f"
server_key_version: "encryptionkey@0"
I20260810 07:15:11.949262 23271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:11.962285 23271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:11.964797 23271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:11.964984 23271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:11.965250 23271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:11.965590 23271 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:11.965646 23271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:11.965688 23271 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:11.965713 23271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:11.971773 23271 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.67:39201
I20260810 07:15:11.971866 23408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.67:39201 every 8 connection(s)
I20260810 07:15:11.972225 23271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
Aug 10 07:15:11 dist-test-slave-10pc krb5kdc[22857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346111, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM
I20260810 07:15:11.978976 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 23271
I20260810 07:15:11.979075 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance
I20260810 07:15:11.979303 16229 external_mini_cluster.cc:1428] Setting key e4429bc5604700d8aedc1e0fdf5ea7cf
I20260810 07:15:11.982267 22957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:11.973038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:48441 (local address 127.15.217.126:41187)
0810 07:15:11.973087 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:11.973094 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:11.973338 (+ 244us) server_negotiation.cc:407] Connection header received
0810 07:15:11.973935 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:11.973936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:11.973951 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:11.974002 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:11.975053 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:11.975574 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:11.976749 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:11.976956 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:11.977065 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:11.979390 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:11.979392 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:11.979393 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:11.979408 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:11.981077 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:11.981459 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:11.981460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:11.981460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:11.981526 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:11.981747 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:11.981747 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:11.981748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:11.981850 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:11.981901 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:11.982139 (+ 238us) server_negotiation.cc:299] Negotiation successful
0810 07:15:11.982203 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 07:15:11.982475 23411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:11.973020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41187 (local address 127.15.217.67:48441)
0810 07:15:11.973208 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:11.973248 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:11.973810 (+ 562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:11.974082 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:11.974085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:11.974692 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:11.974951 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:11.974957 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:11.975691 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:11.975692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:11.976646 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:11.976649 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:11.977075 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:11.977077 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:11.977213 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:11.977715 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:11.977724 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:11.979273 (+ 1549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:11.981148 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:11.981154 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:11.981156 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:11.981392 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:11.981585 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:11.981586 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:11.981586 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:11.981692 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:11.981907 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:11.981912 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:11.982029 (+ 117us) client_negotiation.cc:769] Sending connection context
0810 07:15:11.982112 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 07:15:11.982248 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":90,"threads_started":1}
I20260810 07:15:11.982939 23409 heartbeater.cc:347] Connected to a master server at 127.15.217.126:41187
I20260810 07:15:11.983095 23409 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:11.983304 23409 heartbeater.cc:511] Master 127.15.217.126:41187 requested a full tablet report, sending...
I20260810 07:15:11.983675 22908 ts_manager.cc:194] Registered new tserver with Master: ceed6041617a45b6ab60d5fb2064d12d (127.15.217.67:39201)
I20260810 07:15:11.984014 22908 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.217.67@KRBTEST.COM'} at 127.15.217.67:48441
I20260810 07:15:11.991935 16229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.15.217.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.217.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.217.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.125.keytab.
Entry for principal oryx/127.15.217.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.125.keytab.
I20260810 07:15:12.021324 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.125:44791
--webserver_interface=127.15.217.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.125.keytab
--principal=oryx/127.15.217.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:44299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:41187,127.15.217.125:44791
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:12.117403 23414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:12.117682 23414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:12.117726 23414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:12.119095 23414 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:12.119175 23414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:12.119203 23414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:12.119241 23414 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:12.119273 23414 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:15:12.121125 23414 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:44299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:41187,127.15.217.125:44791
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.125.keytab
--principal=oryx/127.15.217.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.125:44791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.217.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
W20260810 07:15:12.121299 23414 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.15.217.126:41187,127.15.217.125:44791'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260810 07:15:12.121488 23414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:12.121759 23414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:12.127599 23420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:12.127573 23421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:12.127789 23423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:12.127808 23414 server_base.cc:1034] running on GCE node
I20260810 07:15:12.128101 23414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:12.128356 23414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:12.129525 23414 hybrid_clock.cc:648] HybridClock initialized: now 1786346112129516 us; error 38 us; skew 500 ppm
Aug 10 07:15:12 dist-test-slave-10pc krb5kdc[22857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346112, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:12.132444 23414 init.cc:377] Logged in from keytab as oryx/127.15.217.125@KRBTEST.COM (short username oryx)
I20260810 07:15:12.133018 23414 webserver.cc:460] Webserver started at http://127.15.217.125:44781/ using document root <none> and password file <none>
I20260810 07:15:12.133276 23414 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:12.133345 23414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:12.133495 23414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:12.134632 23414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-1/data/instance:
uuid: "be6a5e4564f448cda7ea84840d63d409"
format_stamp: "Formatted at 2026-08-10 07:15:12 on dist-test-slave-10pc"
server_key: "0637e7273799fe1b4396a94f6bc0020b"
server_key_iv: "2514aa1e69ae260b70d0ba03305d88b8"
server_key_version: "encryptionkey@0"
I20260810 07:15:12.135011 23414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-1/wal/instance:
uuid: "be6a5e4564f448cda7ea84840d63d409"
format_stamp: "Formatted at 2026-08-10 07:15:12 on dist-test-slave-10pc"
server_key: "0637e7273799fe1b4396a94f6bc0020b"
server_key_iv: "2514aa1e69ae260b70d0ba03305d88b8"
server_key_version: "encryptionkey@0"
I20260810 07:15:12.136404 23414 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260810 07:15:12.137264 23432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:12.137466 23414 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:15:12.137576 23414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-1/wal
uuid: "be6a5e4564f448cda7ea84840d63d409"
format_stamp: "Formatted at 2026-08-10 07:15:12 on dist-test-slave-10pc"
server_key: "0637e7273799fe1b4396a94f6bc0020b"
server_key_iv: "2514aa1e69ae260b70d0ba03305d88b8"
server_key_version: "encryptionkey@0"
I20260810 07:15:12.137671 23414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:12.156490 23414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:12.159045 23414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:12.159219 23414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:12.163633 23414 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.125:44791
I20260810 07:15:12.163707 23488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.125:44791 every 8 connection(s)
I20260810 07:15:12.164090 23414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-1/data/info.pb
I20260810 07:15:12.165457 23489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:12.167054 23489 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:15:12.168268 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 23414
Aug 10 07:15:12 dist-test-slave-10pc krb5kdc[22857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346112, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.125@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM
Aug 10 07:15:12 dist-test-slave-10pc krb5kdc[22857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346111, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.217.125@KRBTEST.COM
I20260810 07:15:12.178057 22957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.167824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51166 (local address 127.15.217.126:41187)
0810 07:15:12.167889 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:12.167891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:12.168090 (+ 199us) server_negotiation.cc:407] Connection header received
0810 07:15:12.168715 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:12.168717 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:12.168730 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:12.168786 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:12.169614 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.170338 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.171489 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.171689 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.171784 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.174502 (+ 2718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:12.174504 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:12.174504 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:12.174513 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:12.176250 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.176744 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.176745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.176745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.176803 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.177136 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.177137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.177137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.177361 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:12.177455 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:12.177831 (+ 376us) server_negotiation.cc:299] Negotiation successful
0810 07:15:12.177908 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 07:15:12.178444 23494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.168928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48188 (local address 127.15.217.125:44791)
0810 07:15:12.169081 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:12.169085 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:12.169105 (+ 20us) server_negotiation.cc:407] Connection header received
0810 07:15:12.169158 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:12.169161 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:12.169185 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:12.169280 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:12.169857 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.170846 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.171816 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.172221 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.172294 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.175013 (+ 2719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:12.175022 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:12.175023 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:12.175031 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:12.177044 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.177453 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.177456 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.177456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.177515 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.177709 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.177709 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.177710 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.177881 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:12.178024 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:12.178191 (+ 167us) server_negotiation.cc:299] Negotiation successful
0810 07:15:12.178274 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":129,"thread_start_us":81,"threads_started":1}
I20260810 07:15:12.178449 23491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.167733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41187 (local address 127.0.0.1:51166)
0810 07:15:12.167908 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:12.167950 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:12.168594 (+ 644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:12.168864 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:12.168867 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:12.169289 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:12.169520 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.169525 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.170440 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.170441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.171382 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.171385 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.171787 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.171788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.171932 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.172591 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:12.172603 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:12.174406 (+ 1803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:12.176337 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:12.176343 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:12.176346 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:12.176660 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:12.176875 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:12.176876 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:12.176876 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:12.177020 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:12.177448 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:12.177453 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:12.177633 (+ 180us) client_negotiation.cc:769] Sending connection context
0810 07:15:12.177785 (+ 152us) client_negotiation.cc:240] Negotiation successful
0810 07:15:12.177961 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":3,"thread_start_us":94,"threads_started":1}
I20260810 07:15:12.179173 23489 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:15:12 dist-test-slave-10pc krb5kdc[22857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346112, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.125@KRBTEST.COM for oryx/127.15.217.125@KRBTEST.COM
I20260810 07:15:12.185465 23494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.179523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48196 (local address 127.15.217.125:44791)
0810 07:15:12.179591 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:12.179593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:12.179604 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:15:12.179651 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:12.179652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:12.179659 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:12.179702 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:12.179909 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.180599 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.181248 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.181440 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.181501 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.183265 (+ 1764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:12.183266 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:12.183267 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:12.183273 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:12.184444 (+ 1171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.184800 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.184800 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.184801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.184852 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.185027 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.185028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.185028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.185127 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:12.185180 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:12.185285 (+ 105us) server_negotiation.cc:299] Negotiation successful
0810 07:15:12.185390 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":49}
I20260810 07:15:12.185556 23491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.179460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:44791 (local address 127.0.0.1:48196)
0810 07:15:12.179561 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:12.179573 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:12.179613 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:12.179704 (+ 91us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:12.179705 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:12.179815 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:12.179849 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.179850 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.180677 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.180678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.181145 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.181146 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.181509 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.181510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.181553 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.181992 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:12.181997 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:12.183188 (+ 1191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:12.184565 (+ 1377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:12.184566 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:12.184566 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:12.184738 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:12.184912 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:12.184912 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:12.184913 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:12.184975 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:12.185182 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:12.185183 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:12.185218 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 07:15:12.185268 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 07:15:12.185346 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260810 07:15:12.187176 23489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409: Bootstrap starting.
I20260810 07:15:12.188442 23489 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:12.188823 23489 log.cc:824] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:12.189822 23489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409: No bootstrap required, opened a new log
I20260810 07:15:12.191529 23489 raft_consensus.cc:348] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } }
I20260810 07:15:12.191659 23489 raft_consensus.cc:374] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:12.191686 23489 raft_consensus.cc:729] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be6a5e4564f448cda7ea84840d63d409, State: Initialized, Role: FOLLOWER
I20260810 07:15:12.191845 23489 consensus_queue.cc:260] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } }
I20260810 07:15:12.192143 23496 sys_catalog.cc:455] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } } }
I20260810 07:15:12.192221 23489 sys_catalog.cc:564] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:12.192235 23496 sys_catalog.cc:458] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 07:15:12.194108 23509 catalog_manager.cc:1558] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:15:12.194173 23509 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:15:12.196065 23414 master_runner.cc:428] Detected that this master be6a5e4564f448cda7ea84840d63d409 is joining an existing cluster
I20260810 07:15:12.196151 23414 master_runner.cc:432] Initiating AddMaster RPC to add 127.15.217.125:44791
I20260810 07:15:12.203862 22957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.197429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51182 (local address 127.15.217.126:41187)
0810 07:15:12.197478 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:12.197480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:12.197637 (+ 157us) server_negotiation.cc:407] Connection header received
0810 07:15:12.197703 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:12.197704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:12.197714 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:12.197769 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:12.198180 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.198688 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.199443 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.199655 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.199740 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.200881 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:12.200882 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:12.200882 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:12.200890 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:12.202536 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.202962 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.202963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.202964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.203030 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.203246 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.203247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.203247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.203365 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:12.203448 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:12.203666 (+ 218us) server_negotiation.cc:299] Negotiation successful
0810 07:15:12.203748 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 07:15:12.203876 23516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.197231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41187 (local address 127.0.0.1:51182)
0810 07:15:12.197581 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:12.197587 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:12.197644 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:12.197776 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:12.197776 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:12.198039 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:12.198099 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.198101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.198773 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.198774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.199354 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.199356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.199757 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.199758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.199828 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.200322 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:12.200327 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:12.200769 (+ 442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:12.202636 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:12.202637 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:12.202637 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:12.202874 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:12.203101 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:12.203102 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:12.203102 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:12.203168 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:12.203476 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:12.203477 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:12.203534 (+ 57us) client_negotiation.cc:769] Sending connection context
0810 07:15:12.203636 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 07:15:12.203751 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":5,"thread_start_us":93,"threads_started":1}
I20260810 07:15:12.204308 23517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.197231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:44791 (local address 127.0.0.1:48212)
0810 07:15:12.197414 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:12.197425 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:12.197497 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:12.197695 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:12.197696 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:12.197992 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:12.198094 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.198097 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.198993 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.198993 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.199849 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.199852 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.200323 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.200324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.200401 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.200846 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:12.200851 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:12.201249 (+ 398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:12.203132 (+ 1883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:12.203133 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:12.203133 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:12.203335 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:12.203539 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:12.203540 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:12.203540 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:12.203613 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:12.203921 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:12.203922 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:12.204013 (+ 91us) client_negotiation.cc:769] Sending connection context
0810 07:15:12.204084 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 07:15:12.204189 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":1,"spinlock_wait_cycles":11136,"thread_start_us":116,"threads_started":1}
I20260810 07:15:12.204366 23494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.197371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48212 (local address 127.15.217.125:44791)
0810 07:15:12.197413 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:12.197416 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:12.197472 (+ 56us) server_negotiation.cc:407] Connection header received
0810 07:15:12.197571 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:12.197572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:12.197583 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:12.197646 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:12.198181 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.198903 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.199940 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.200213 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.200253 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.201345 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:12.201346 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:12.201346 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:12.201352 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:12.203042 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.203408 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.203409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.203409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.203462 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.203704 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.203704 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.203705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.203838 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:12.203880 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:12.204137 (+ 257us) server_negotiation.cc:299] Negotiation successful
0810 07:15:12.204245 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":25}
Aug 10 07:15:12 dist-test-slave-10pc krb5kdc[22857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346111, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.126@KRBTEST.COM for oryx/127.15.217.125@KRBTEST.COM
I20260810 07:15:12.214553 23494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.206824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48220 (local address 127.15.217.125:44791)
0810 07:15:12.206864 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:12.206866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:12.207015 (+ 149us) server_negotiation.cc:407] Connection header received
0810 07:15:12.207118 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:12.207119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:12.207127 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:12.207171 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:12.207697 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.208159 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.209365 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.209542 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.209597 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.211860 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:12.211862 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:12.211862 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:12.211870 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:12.213297 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.213713 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.213714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.213715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.213775 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.213982 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.213982 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.213983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.214102 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:12.214182 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:12.214368 (+ 186us) server_negotiation.cc:299] Negotiation successful
0810 07:15:12.214449 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 07:15:12.214610 23518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.206741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:44791 (local address 127.0.0.1:48220)
0810 07:15:12.206899 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:12.206956 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:12.207038 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:12.207178 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:12.207181 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:12.207544 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:12.207608 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.207610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.208237 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.208239 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.209270 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.209273 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.209619 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.209620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.209737 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.210163 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:12.210168 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:12.211766 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:12.213393 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:12.213402 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:12.213405 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:12.213634 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:12.213845 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:12.213846 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:12.213846 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:12.213916 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:12.214174 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:12.214183 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:12.214238 (+ 55us) client_negotiation.cc:769] Sending connection context
0810 07:15:12.214340 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 07:15:12.214464 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
I20260810 07:15:12.215199 22908 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.15.217.125:44791: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "be6a5e4564f448cda7ea84840d63d409"
member_type: NON_VOTER
last_known_addr {
host: "127.15.217.125"
port: 44791
}
attrs {
promote: true
}
}
dest_uuid: "29e10cc613d84f3e807c6339559f7930"
cas_config_opid_index: -1
I20260810 07:15:12.215546 22908 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: NON_VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: true } }
I20260810 07:15:12.216077 23520 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER be6a5e4564f448cda7ea84840d63d409 (127.15.217.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: NON_VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: true } } }
I20260810 07:15:12.216414 23460 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:12.217118 23520 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.15.217.125:44791
I20260810 07:15:12.217274 23519 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "29e10cc613d84f3e807c6339559f7930" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: NON_VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: true } } }
I20260810 07:15:12.217484 23519 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:12.217482 23460 raft_consensus.cc:1261] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [term 1 FOLLOWER]: Refusing update from remote peer 29e10cc613d84f3e807c6339559f7930: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260810 07:15:12.217495 23414 master.cc:537] Master@127.15.217.125:44791 shutting down...
I20260810 07:15:12.218086 23519 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: NON_VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 07:15:12.218864 22894 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260810 07:15:12.220095 22894 consensus_peers.cc:487] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 -> Peer be6a5e4564f448cda7ea84840d63d409 (127.15.217.125:44791): Couldn't send request to peer be6a5e4564f448cda7ea84840d63d409. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:15:12.220764 23414 raft_consensus.cc:2229] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:12.220856 23414 raft_consensus.cc:2258] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:12.220894 23414 tablet_replica.cc:330] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409: stopping tablet replica
I20260810 07:15:12.233611 23414 master.cc:552] Master@127.15.217.125:44791 shutdown complete.
I20260810 07:15:12.233664 23414 master_runner.cc:305] Clearing existing system tablet
I20260810 07:15:12.233817 23414 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:12.233865 23414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:12.234517 23414 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.002s
I20260810 07:15:12.235028 23524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:12.235153 23414 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:12.235210 23414 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 07:15:12.235232 23414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-1/wal
uuid: "be6a5e4564f448cda7ea84840d63d409"
format_stamp: "Formatted at 2026-08-10 07:15:12 on dist-test-slave-10pc"
server_key: "0637e7273799fe1b4396a94f6bc0020b"
server_key_iv: "2514aa1e69ae260b70d0ba03305d88b8"
server_key_version: "encryptionkey@0"
I20260810 07:15:12.235976 23414 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:12.237636 23414 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 07:15:12.237722 23414 log.cc:1196] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260810 07:15:12.237938 23414 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409: Deleting consensus metadata
I20260810 07:15:12.238090 23414 master_runner.cc:315] Copying system tablet from 127.15.217.126:41187
I20260810 07:15:12.238790 23414 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409: tablet copy: Beginning tablet copy session from remote peer at address 127.15.217.126:41187
I20260810 07:15:12.247179 23530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.239165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41187 (local address 127.0.0.1:51190)
0810 07:15:12.239388 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:12.239401 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:12.239524 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:12.239713 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:12.239715 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:12.239989 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:12.240081 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.240083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.241115 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.241116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.242090 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.242094 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.242547 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.242548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.242626 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.243576 (+ 950us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:12.243581 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:12.243888 (+ 307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:12.246116 (+ 2228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:12.246117 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:12.246117 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:12.246307 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:12.246541 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:12.246542 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:12.246542 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:12.246597 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:12.246902 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:12.246903 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:12.246943 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 07:15:12.246993 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 07:15:12.247094 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":71,"threads_started":1}
I20260810 07:15:12.247539 22957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.239282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51190 (local address 127.15.217.126:41187)
0810 07:15:12.239340 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:12.239342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:12.239491 (+ 149us) server_negotiation.cc:407] Connection header received
0810 07:15:12.239607 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:12.239609 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:12.239621 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:12.239705 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:12.240172 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.241021 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.242237 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.242449 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.242545 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.243987 (+ 1442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:12.243989 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:12.243989 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:12.243998 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:12.246001 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.246389 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.246390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.246390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.246465 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.246658 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.246659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.246659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.246826 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:12.246899 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:12.247045 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 07:15:12.247137 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 07:15:12.247993 22933 tablet_copy_service.cc:140] P 29e10cc613d84f3e807c6339559f7930: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer be6a5e4564f448cda7ea84840d63d409 ({username='oryx', principal='oryx/127.15.217.125@KRBTEST.COM'} at 127.0.0.1:51190)
I20260810 07:15:12.248147 22933 tablet_copy_service.cc:161] P 29e10cc613d84f3e807c6339559f7930: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer be6a5e4564f448cda7ea84840d63d409 at {username='oryx', principal='oryx/127.15.217.125@KRBTEST.COM'} at 127.0.0.1:51190: session id = be6a5e4564f448cda7ea84840d63d409-00000000000000000000000000000000
I20260810 07:15:12.249258 22933 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 07:15:12.249984 23414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 07:15:12.251087 23414 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409: tablet copy: Starting download of 0 data blocks...
I20260810 07:15:12.251276 23414 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409: tablet copy: Starting download of 1 WAL segments...
I20260810 07:15:12.252532 23414 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 07:15:12.253404 22933 tablet_copy_service.cc:342] P 29e10cc613d84f3e807c6339559f7930: Request end of tablet copy session be6a5e4564f448cda7ea84840d63d409-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.15.217.125@KRBTEST.COM'} at 127.0.0.1:51190
I20260810 07:15:12.253474 22933 tablet_copy_service.cc:433] P 29e10cc613d84f3e807c6339559f7930: ending tablet copy session be6a5e4564f448cda7ea84840d63d409-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer be6a5e4564f448cda7ea84840d63d409
I20260810 07:15:12.255823 23414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:12.257037 23532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:12.257048 23531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:12.257124 23534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:12.257416 23414 server_base.cc:1034] running on GCE node
I20260810 07:15:12.257510 23414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:12.257649 23414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:12.258787 23414 hybrid_clock.cc:648] HybridClock initialized: now 1786346112258788 us; error 35 us; skew 500 ppm
Aug 10 07:15:12 dist-test-slave-10pc krb5kdc[22857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346112, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:12.260398 23414 init.cc:377] Logged in from keytab as oryx/127.15.217.125@KRBTEST.COM (short username oryx)
I20260810 07:15:12.260732 23414 webserver.cc:460] Webserver started at http://127.15.217.125:33801/ using document root <none> and password file <none>
I20260810 07:15:12.260875 23414 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:12.260916 23414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:12.261543 23414 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260810 07:15:12.262085 23542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:12.262207 23414 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 07:15:12.262255 23414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-1/wal
uuid: "be6a5e4564f448cda7ea84840d63d409"
format_stamp: "Formatted at 2026-08-10 07:15:12 on dist-test-slave-10pc"
server_key: "0637e7273799fe1b4396a94f6bc0020b"
server_key_iv: "2514aa1e69ae260b70d0ba03305d88b8"
server_key_version: "encryptionkey@0"
I20260810 07:15:12.262316 23414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:12.279372 23414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:12.281324 23414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:12.284479 23414 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.125:44791
I20260810 07:15:12.284703 23599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.125:44791 every 8 connection(s)
I20260810 07:15:12.284874 23414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-1/data/info.pb
I20260810 07:15:12.285633 23600 sys_catalog.cc:263] Verifying existing consensus state
I20260810 07:15:12.286052 23600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409: Bootstrap starting.
I20260810 07:15:12.288421 23600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:15:12.288676 23600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409: Bootstrap complete.
I20260810 07:15:12.288901 23600 raft_consensus.cc:348] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: NON_VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: true } }
I20260810 07:15:12.288992 23600 raft_consensus.cc:729] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: be6a5e4564f448cda7ea84840d63d409, State: Initialized, Role: LEARNER
I20260810 07:15:12.289074 23600 consensus_queue.cc:260] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: NON_VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: true } }
I20260810 07:15:12.289250 23600 sys_catalog.cc:564] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:12.289296 23605 sys_catalog.cc:455] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: NON_VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: true } } }
I20260810 07:15:12.289393 23605 sys_catalog.cc:458] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [sys.catalog]: This master's current role is: LEARNER
I20260810 07:15:12.290774 23618 catalog_manager.cc:1259] Loaded cluster ID: 213dbfc3d5314eb3b1002679033b6e06
I20260810 07:15:12.290982 23618 catalog_manager.cc:1552] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409: loading cluster ID for follower catalog manager: success
I20260810 07:15:12.292009 23618 catalog_manager.cc:1574] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409: acquiring CA information for follower catalog manager: success
I20260810 07:15:12.292757 23618 catalog_manager.cc:1602] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:15:12.338171 23619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.331201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48258 (local address 127.15.217.125:44791)
0810 07:15:12.331378 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:12.331379 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:12.331387 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:15:12.331413 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:12.331414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:12.331424 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:12.331497 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:12.332130 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.332907 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.333685 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.333900 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.333981 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.335293 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:12.335295 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:12.335295 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:12.335303 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:12.336908 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.337323 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.337325 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.337325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.337391 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.337613 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.337614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.337614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.337738 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:12.337848 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:12.337950 (+ 102us) server_negotiation.cc:299] Negotiation successful
0810 07:15:12.338054 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":93,"threads_started":1}
WARNING: no policy specified for oryx/127.15.217.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.217.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.217.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.124.keytab.
Entry for principal oryx/127.15.217.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.124.keytab.
I20260810 07:15:12.368108 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.124:41403
--webserver_interface=127.15.217.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.124.keytab
--principal=oryx/127.15.217.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:44299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:41187,127.15.217.125:44791,127.15.217.124:41403
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:12.476557 23622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:12.476831 23622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:12.476878 23622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:12.478334 23622 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:12.478415 23622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:12.478437 23622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:12.478457 23622 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:12.478485 23622 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:15:12.480522 23622 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:44299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:41187,127.15.217.125:44791,127.15.217.124:41403
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.124.keytab
--principal=oryx/127.15.217.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.124:41403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.217.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:12.480854 23622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:12.481132 23622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:12.487083 23628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:12.487102 23630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:12.487120 23622 server_base.cc:1034] running on GCE node
W20260810 07:15:12.487080 23627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:12.487658 23622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:12.487987 23622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:12.489149 23622 hybrid_clock.cc:648] HybridClock initialized: now 1786346112489134 us; error 31 us; skew 500 ppm
Aug 10 07:15:12 dist-test-slave-10pc krb5kdc[22857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346112, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:12.493300 23622 init.cc:377] Logged in from keytab as oryx/127.15.217.124@KRBTEST.COM (short username oryx)
I20260810 07:15:12.493801 23622 webserver.cc:460] Webserver started at http://127.15.217.124:32859/ using document root <none> and password file <none>
I20260810 07:15:12.494050 23622 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:12.494122 23622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:12.494274 23622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:12.495710 23622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-2/data/instance:
uuid: "c2d2d5a77d9f482f93cc231f8b7e5324"
format_stamp: "Formatted at 2026-08-10 07:15:12 on dist-test-slave-10pc"
server_key: "bb9189314ca46215356fb8b88297af52"
server_key_iv: "8fa66e873f993fcb4bfa4454aded3e28"
server_key_version: "encryptionkey@0"
I20260810 07:15:12.496151 23622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-2/wal/instance:
uuid: "c2d2d5a77d9f482f93cc231f8b7e5324"
format_stamp: "Formatted at 2026-08-10 07:15:12 on dist-test-slave-10pc"
server_key: "bb9189314ca46215356fb8b88297af52"
server_key_iv: "8fa66e873f993fcb4bfa4454aded3e28"
server_key_version: "encryptionkey@0"
I20260810 07:15:12.497887 23622 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260810 07:15:12.498788 23638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:12.498975 23622 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:12.499090 23622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-2/wal
uuid: "c2d2d5a77d9f482f93cc231f8b7e5324"
format_stamp: "Formatted at 2026-08-10 07:15:12 on dist-test-slave-10pc"
server_key: "bb9189314ca46215356fb8b88297af52"
server_key_iv: "8fa66e873f993fcb4bfa4454aded3e28"
server_key_version: "encryptionkey@0"
I20260810 07:15:12.499167 23622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:12.514285 23622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:12.517005 23622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:12.517254 23622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:12.528359 23622 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.124:41403
I20260810 07:15:12.528594 23694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.124:41403 every 8 connection(s)
I20260810 07:15:12.528966 23622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-2/data/info.pb
I20260810 07:15:12.530337 23695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:12.532789 23695 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:15:12.535172 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 23622
Aug 10 07:15:12 dist-test-slave-10pc krb5kdc[22857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346112, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.124@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM
Aug 10 07:15:12 dist-test-slave-10pc krb5kdc[22857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346111, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.217.124@KRBTEST.COM
I20260810 07:15:12.544206 22957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.533808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51200 (local address 127.15.217.126:41187)
0810 07:15:12.533885 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:12.533887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:12.534071 (+ 184us) server_negotiation.cc:407] Connection header received
0810 07:15:12.534787 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:12.534789 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:12.534798 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:12.534863 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:12.535857 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.536585 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.537855 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.538187 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.538292 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.541225 (+ 2933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:12.541226 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:12.541227 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:12.541236 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:12.542765 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.543214 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.543215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.543216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.543270 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.543531 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.543532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.543533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.543655 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:12.543729 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:12.544023 (+ 294us) server_negotiation.cc:299] Negotiation successful
0810 07:15:12.544110 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 07:15:12.544445 23697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.533647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41187 (local address 127.0.0.1:51200)
0810 07:15:12.533916 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:12.533968 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:12.534694 (+ 726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:12.534921 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:12.534924 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:12.535238 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:12.535684 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.535692 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.536808 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.536810 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.537755 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.537757 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.538454 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.538455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.538645 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.539294 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:12.539306 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:12.541128 (+ 1822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:12.542855 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:12.542862 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:12.542865 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:12.543117 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:12.543337 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:12.543338 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:12.543339 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:12.543450 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:12.543731 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:12.543739 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:12.543879 (+ 140us) client_negotiation.cc:769] Sending connection context
0810 07:15:12.543987 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 07:15:12.544160 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":92,"threads_started":1}
I20260810 07:15:12.545228 23695 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:15:12.546103 23698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.535736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34836 (local address 127.15.217.124:41403)
0810 07:15:12.536009 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:12.536013 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:12.536027 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:15:12.536053 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:12.536054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:12.536088 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:12.536178 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:12.537012 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.538240 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.539346 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.539693 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.539752 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.542527 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:12.542535 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:12.542535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:12.542542 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:12.544515 (+ 1973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.545017 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.545020 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.545020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.545083 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.545298 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.545299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.545299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.545479 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:12.545856 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:12.545906 (+ 50us) server_negotiation.cc:299] Negotiation successful
0810 07:15:12.545992 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":154,"threads_started":1}
Aug 10 07:15:12 dist-test-slave-10pc krb5kdc[22857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346112, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.124@KRBTEST.COM for oryx/127.15.217.125@KRBTEST.COM
I20260810 07:15:12.552131 23697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.545522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:44791 (local address 127.0.0.1:48270)
0810 07:15:12.545582 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:12.545593 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:12.545633 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:12.545817 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:12.545818 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:12.546018 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:12.546170 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.546173 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.546954 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.546955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.547475 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.547476 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.547833 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.547834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.547880 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.548225 (+ 345us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:12.548228 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:12.549437 (+ 1209us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:12.551107 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:12.551108 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:12.551108 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:12.551268 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:12.551449 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:12.551450 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:12.551450 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:12.551517 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:12.551827 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:12.551828 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:12.551905 (+ 77us) client_negotiation.cc:769] Sending connection context
0810 07:15:12.551978 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 07:15:12.552065 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 07:15:12.552161 23619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.545624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48270 (local address 127.15.217.125:44791)
0810 07:15:12.545681 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:12.545683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:12.545692 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:15:12.545719 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:12.545720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:12.545736 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:12.545811 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:12.546254 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.546886 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.547561 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.547766 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.547818 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.549522 (+ 1704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:12.549523 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:12.549524 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:12.549531 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:12.551035 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.551331 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.551332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.551332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.551383 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.551591 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.551592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.551592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.551763 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:12.551818 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:12.552010 (+ 192us) server_negotiation.cc:299] Negotiation successful
0810 07:15:12.552096 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 07:15:12.552724 23695 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.217.124" port: 41403 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:15:12 dist-test-slave-10pc krb5kdc[22857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346112, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.124@KRBTEST.COM for oryx/127.15.217.124@KRBTEST.COM
I20260810 07:15:12.559307 23698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.553101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34844 (local address 127.15.217.124:41403)
0810 07:15:12.553150 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:12.553152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:12.553160 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:15:12.553179 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:12.553180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:12.553196 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:12.553239 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:12.553451 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.553993 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.554645 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.554932 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.554986 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.556926 (+ 1940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:12.556927 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:12.556928 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:12.556934 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:12.558419 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.558733 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.558734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.558734 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.558781 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.558929 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.558930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.558930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.559022 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:12.559069 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:12.559176 (+ 107us) server_negotiation.cc:299] Negotiation successful
0810 07:15:12.559241 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 07:15:12.559449 23697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.553022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:41403 (local address 127.0.0.1:34844)
0810 07:15:12.553059 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:12.553070 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:12.553110 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:12.553254 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:12.553255 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:12.553363 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:12.553391 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.553392 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.554124 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.554125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.554576 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.554577 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.555013 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.555014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.555053 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.555376 (+ 323us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:12.555380 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:12.556844 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:12.558501 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:12.558502 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:12.558503 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:12.558670 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:12.558837 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:12.558838 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:12.558838 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:12.558890 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:12.559072 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:12.559073 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:12.559110 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 07:15:12.559160 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 07:15:12.559288 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19,"mutex_wait_us":47}
I20260810 07:15:12.561335 23695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324: Bootstrap starting.
I20260810 07:15:12.562602 23695 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:12.563269 23695 log.cc:824] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:12.564325 23695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324: No bootstrap required, opened a new log
I20260810 07:15:12.567241 23695 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 41403 } }
I20260810 07:15:12.567404 23695 raft_consensus.cc:374] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:12.567454 23695 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2d2d5a77d9f482f93cc231f8b7e5324, State: Initialized, Role: FOLLOWER
I20260810 07:15:12.567608 23695 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 41403 } }
I20260810 07:15:12.568007 23695 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:12.568089 23701 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 41403 } } }
I20260810 07:15:12.568248 23701 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 07:15:12.570796 23712 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:15:12.570873 23712 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:15:12.573267 23622 master_runner.cc:428] Detected that this master c2d2d5a77d9f482f93cc231f8b7e5324 is joining an existing cluster
I20260810 07:15:12.573329 23622 master_runner.cc:432] Initiating AddMaster RPC to add 127.15.217.124:41403
I20260810 07:15:12.583693 23619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.574839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48284 (local address 127.15.217.125:44791)
0810 07:15:12.574890 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:12.574893 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:12.575473 (+ 580us) server_negotiation.cc:407] Connection header received
0810 07:15:12.575557 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:12.575558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:12.575569 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:12.575614 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:12.576353 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.577347 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.578869 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.579181 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.579237 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.580656 (+ 1419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:12.580658 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:12.580658 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:12.580665 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:12.582272 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.582705 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.582706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.582707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.582772 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.583104 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.583106 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.583106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.583253 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:12.583345 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:12.583504 (+ 159us) server_negotiation.cc:299] Negotiation successful
0810 07:15:12.583595 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 07:15:12.583768 23718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.574965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:44791 (local address 127.0.0.1:48284)
0810 07:15:12.575405 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:12.575417 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:12.575485 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:12.575686 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:12.575687 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:12.575981 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:12.576257 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.576261 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.577452 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.577453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.578773 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.578776 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.579360 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.579361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.579428 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.580072 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:12.580076 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:12.580561 (+ 485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:12.582362 (+ 1801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:12.582363 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:12.582364 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:12.582613 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:12.582842 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:12.582843 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:12.582843 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:12.582974 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:12.583337 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:12.583338 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:12.583386 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 07:15:12.583477 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 07:15:12.583595 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":280,"threads_started":1}
I20260810 07:15:12.585341 22957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.574841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51216 (local address 127.15.217.126:41187)
0810 07:15:12.574908 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:12.574911 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:12.576275 (+ 1364us) server_negotiation.cc:407] Connection header received
0810 07:15:12.576397 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:12.576398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:12.576411 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:12.576535 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:12.577311 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.578136 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.580313 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.580628 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.580707 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.582171 (+ 1464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:12.582172 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:12.582172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:12.582181 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:12.583845 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.584563 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.584563 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.584564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.584620 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.584835 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.584836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.584836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.584948 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:12.584999 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:12.585211 (+ 212us) server_negotiation.cc:299] Negotiation successful
0810 07:15:12.585278 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 07:15:12.585685 23719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.575526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41187 (local address 127.0.0.1:51216)
0810 07:15:12.576189 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:12.576201 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:12.576305 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:12.576577 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:12.576579 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:12.576916 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:12.577212 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.577215 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.578236 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.578237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.580214 (+ 1977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.580217 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.580775 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.580776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.580897 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.581445 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:12.581451 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:12.582070 (+ 619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:12.584166 (+ 2096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:12.584167 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:12.584167 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:12.584446 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:12.584688 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:12.584689 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:12.584689 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:12.584769 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:12.585037 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:12.585038 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:12.585095 (+ 57us) client_negotiation.cc:769] Sending connection context
0810 07:15:12.585189 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 07:15:12.585314 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"mutex_wait_us":137,"thread_start_us":116,"threads_started":1}
Aug 10 07:15:12 dist-test-slave-10pc krb5kdc[22857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346111, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.126@KRBTEST.COM for oryx/127.15.217.124@KRBTEST.COM
I20260810 07:15:12.597354 23518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.588295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:41403 (local address 127.0.0.1:34860)
0810 07:15:12.588364 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:12.588377 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:12.588423 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:12.588878 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:12.588879 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:12.589007 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:12.589080 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.589082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.590330 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.590331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.591091 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.591094 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.591756 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.591757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.591821 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.592250 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:12.592256 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:12.593617 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:12.596063 (+ 2446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:12.596064 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:12.596065 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:12.596281 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:12.596523 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:12.596524 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:12.596524 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:12.596602 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:12.597019 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:12.597021 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:12.597070 (+ 49us) client_negotiation.cc:769] Sending connection context
0810 07:15:12.597121 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 07:15:12.597248 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 07:15:12.597384 23698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.575905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34854 (local address 127.15.217.124:41403)
0810 07:15:12.575946 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:12.575948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:12.576558 (+ 610us) server_negotiation.cc:407] Connection header received
0810 07:15:12.576639 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:12.576639 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:12.576647 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:12.576672 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:12.577332 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.578172 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.579419 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.579698 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.579731 (+ 33us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.594261 (+ 14530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:12.594263 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:12.594263 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:12.594270 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:12.596175 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.596577 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.596578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.596578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.596623 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.596874 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.596875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.596875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.597000 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:12.597070 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:12.597227 (+ 157us) server_negotiation.cc:299] Negotiation successful
0810 07:15:12.597297 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 07:15:12.597532 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 22981
I20260810 07:15:12.597894 23720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.575979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:41403 (local address 127.0.0.1:34854)
0810 07:15:12.576496 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:12.576505 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:12.576579 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:12.576720 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:12.576720 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:12.577073 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:12.577250 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.577252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.578259 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.578260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.579328 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.579331 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.579782 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.579783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.579880 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.593762 (+ 13882us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:12.593767 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:12.594105 (+ 338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:12.596258 (+ 2153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:12.596259 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:12.596259 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:12.596492 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:12.596685 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:12.596686 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:12.596687 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:12.596741 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:12.597094 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:12.597095 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:12.597132 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 07:15:12.597660 (+ 528us) client_negotiation.cc:240] Negotiation successful
0810 07:15:12.597744 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"mutex_wait_us":40,"thread_start_us":288,"threads_started":1}
I20260810 07:15:12.598428 23721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.588333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34860 (local address 127.15.217.124:41403)
0810 07:15:12.588711 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:12.588713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:12.588721 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:15:12.588780 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:12.588781 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:12.588792 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:12.588861 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:12.589225 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.590226 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.591167 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.591670 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.591757 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.593835 (+ 2078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:12.593838 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:12.593838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:12.593845 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:12.595972 (+ 2127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.596355 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.596356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.596357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.596427 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.596655 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.596656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.596657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.596928 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:12.597589 (+ 661us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:12.597638 (+ 49us) server_negotiation.cc:299] Negotiation successful
0810 07:15:12.598324 (+ 686us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":252,"threads_started":1}
I20260810 07:15:12.599036 22908 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.15.217.124:41403: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324"
member_type: NON_VOTER
last_known_addr {
host: "127.15.217.124"
port: 41403
}
attrs {
promote: true
}
}
dest_uuid: "29e10cc613d84f3e807c6339559f7930"
cas_config_opid_index: 5
I20260810 07:15:12.599334 22908 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: NON_VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: true } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: NON_VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: true } }
I20260810 07:15:12.600050 23668 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:12.601544 23668 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [term 1 FOLLOWER]: Refusing update from remote peer 29e10cc613d84f3e807c6339559f7930: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 07:15:12.601850 23520 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: NON_VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:12.603546 23668 raft_consensus.cc:858] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 41403 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: NON_VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: NON_VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: true } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: NON_VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: true } }
W20260810 07:15:12.603624 23668 raft_consensus.cc:1565] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 41403 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: NON_VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: NON_VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: true } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: NON_VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: true } }
I20260810 07:15:12.604326 23668 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER be6a5e4564f448cda7ea84840d63d409 (127.15.217.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: NON_VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: true } } }
I20260810 07:15:12.606066 23726 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "29e10cc613d84f3e807c6339559f7930" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: NON_VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: true } } }
I20260810 07:15:12.606185 23701 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29e10cc613d84f3e807c6339559f7930. Latest consensus state: current_term: 1 leader_uuid: "29e10cc613d84f3e807c6339559f7930" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: NON_VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: true } } }
I20260810 07:15:12.606280 23726 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260810 07:15:12.606415 23726 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "29e10cc613d84f3e807c6339559f7930" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: NON_VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: true } } }
I20260810 07:15:12.606516 23726 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260810 07:15:12.606706 23520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "29e10cc613d84f3e807c6339559f7930" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: NON_VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: true } } } pending_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: NON_VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: true } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: NON_VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: true } } }
I20260810 07:15:12.606818 23520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:12.606297 23701 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260810 07:15:12.607118 23668 raft_consensus.cc:1467] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [term 1 NON_PARTICIPANT]: Allowing update even though not a member of the config
I20260810 07:15:12.607458 23520 raft_consensus.cc:1013] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [term 1 LEADER]: attempt to promote peer c2d2d5a77d9f482f93cc231f8b7e5324: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260810 07:15:12.609354 23519 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER c2d2d5a77d9f482f93cc231f8b7e5324 (127.15.217.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: NON_VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: true } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: NON_VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: true } } }
I20260810 07:15:12.610515 23519 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.15.217.124:41403
I20260810 07:15:12.610674 23519 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "29e10cc613d84f3e807c6339559f7930" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: NON_VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: true } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: NON_VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: true } } }
I20260810 07:15:12.610744 23519 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:12.611353 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.65:34201
--local_ip_for_outbound_sockets=127.15.217.65
--tserver_master_addrs=127.15.217.126:41187
--webserver_port=41719
--webserver_interface=127.15.217.65
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.65.keytab
--principal=oryx/127.15.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:44299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.217.126:41187,127.15.217.125:44791
--tserver_master_addrs=127.15.217.126:41187,127.15.217.125:44791,127.15.217.124:41403 with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:15:12.611843 23622 master.cc:537] Master@127.15.217.124:41403 shutting down...
W20260810 07:15:12.612849 22891 consensus_peers.cc:487] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 -> Peer c2d2d5a77d9f482f93cc231f8b7e5324 (127.15.217.124:41403): Couldn't send request to peer c2d2d5a77d9f482f93cc231f8b7e5324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:15:12.612852 23518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.599845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:44791 (local address 127.0.0.1:48292)
0810 07:15:12.602246 (+ 2401us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:12.602279 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:12.602317 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:12.608991 (+ 6674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:12.608993 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:12.608995 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:12.609049 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.609051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.610493 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.610494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.611555 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.611559 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.612551 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.612552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.612624 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.612653 (+ 29us) client_negotiation.cc:769] Sending connection context
0810 07:15:12.612710 (+ 57us) client_negotiation.cc:240] Negotiation successful
0810 07:15:12.612767 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2381}
I20260810 07:15:12.613026 23619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.608692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48292 (local address 127.15.217.125:44791)
0810 07:15:12.608755 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:12.608758 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:12.608770 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:15:12.608802 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:12.608804 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:12.608834 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:12.608907 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:12.609138 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.609888 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.611668 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.612300 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.612390 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.612512 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:12.612884 (+ 372us) server_negotiation.cc:299] Negotiation successful
0810 07:15:12.612938 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 07:15:12.613394 23570 raft_consensus.cc:1261] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [term 1 LEARNER]: Refusing update from remote peer 29e10cc613d84f3e807c6339559f7930: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 07:15:12.613659 23727 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: NON_VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260810 07:15:12.614445 23570 raft_consensus.cc:2932] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER c2d2d5a77d9f482f93cc231f8b7e5324 (127.15.217.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: NON_VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: true } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: NON_VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: true } } }
I20260810 07:15:12.614880 23723 mvcc.cc:205] Tried to move back new op lower bound from 7316873672501612544 to 7316873672384487424. Current Snapshot: MvccSnapshot[applied={T|T < 7316873672398020608}]
I20260810 07:15:12.616052 23605 sys_catalog.cc:455] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29e10cc613d84f3e807c6339559f7930. Latest consensus state: current_term: 1 leader_uuid: "29e10cc613d84f3e807c6339559f7930" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: NON_VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: true } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: NON_VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: true } } }
I20260810 07:15:12.616154 23605 sys_catalog.cc:458] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [sys.catalog]: This master's current role is: LEARNER
I20260810 07:15:12.616351 23727 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930: attempting to promote NON_VOTER be6a5e4564f448cda7ea84840d63d409 to VOTER
I20260810 07:15:12.616655 23727 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: false } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: NON_VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: true } }
I20260810 07:15:12.617179 23570 raft_consensus.cc:1261] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [term 1 LEARNER]: Refusing update from remote peer 29e10cc613d84f3e807c6339559f7930: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260810 07:15:12.617358 23520 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260810 07:15:12.617482 22891 consensus_peers.cc:487] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 -> Peer c2d2d5a77d9f482f93cc231f8b7e5324 (127.15.217.124:41403): Couldn't send request to peer c2d2d5a77d9f482f93cc231f8b7e5324. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:15:12.617607 23622 raft_consensus.cc:2229] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [term 1 LEARNER]: Raft consensus shutting down.
I20260810 07:15:12.617697 23622 pending_rounds.cc:62] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324: Trying to abort 1 pending ops.
I20260810 07:15:12.617736 23622 pending_rounds.cc:68] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324: Aborting op as it isn't in flight: id { term: 1 index: 6 } timestamp: 7316873677206188032 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "00000000000000000000000000000000" old_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: NON_VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: true } } } new_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: NON_VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: true } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: NON_VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: true } } } }
I20260810 07:15:12.617841 23622 raft_consensus.cc:2894] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [term 1 LEARNER]: Aborting config change with OpId 1.6: Aborted: Op aborted
I20260810 07:15:12.617875 23622 raft_consensus.cc:2866] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [term 1 NON_PARTICIPANT]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260810 07:15:12.617898 23622 raft_consensus.cc:2258] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260810 07:15:12.617929 23622 tablet_replica.cc:330] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324: stopping tablet replica
I20260810 07:15:12.618008 23520 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, be6a5e4564f448cda7ea84840d63d409 (127.15.217.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: false } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: NON_VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: true } } }
I20260810 07:15:12.618216 23735 sys_catalog.cc:455] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "29e10cc613d84f3e807c6339559f7930" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: NON_VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: true } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: NON_VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: true } } }
I20260810 07:15:12.618336 23735 sys_catalog.cc:458] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [sys.catalog]: This master's current role is: LEARNER
I20260810 07:15:12.618865 23520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "29e10cc613d84f3e807c6339559f7930" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: false } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: NON_VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: true } } }
I20260810 07:15:12.618956 23520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:12.619017 23520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "29e10cc613d84f3e807c6339559f7930" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: false } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: NON_VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: true } } }
I20260810 07:15:12.619066 23520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:12.619287 23570 raft_consensus.cc:2932] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, be6a5e4564f448cda7ea84840d63d409 (127.15.217.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: false } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: NON_VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: true } } }
I20260810 07:15:12.620059 23735 sys_catalog.cc:455] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "29e10cc613d84f3e807c6339559f7930" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: false } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: NON_VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: true } } }
I20260810 07:15:12.620214 23735 sys_catalog.cc:458] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:15:12.634646 23622 master.cc:552] Master@127.15.217.124:41403 shutdown complete.
I20260810 07:15:12.634723 23622 master_runner.cc:305] Clearing existing system tablet
I20260810 07:15:12.634873 23622 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:12.634939 23622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:12.635622 23622 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.001s
I20260810 07:15:12.636196 23737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:12.636395 23622 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:12.636539 23622 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:12.636577 23622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-2/wal
uuid: "c2d2d5a77d9f482f93cc231f8b7e5324"
format_stamp: "Formatted at 2026-08-10 07:15:12 on dist-test-slave-10pc"
server_key: "bb9189314ca46215356fb8b88297af52"
server_key_iv: "8fa66e873f993fcb4bfa4454aded3e28"
server_key_version: "encryptionkey@0"
I20260810 07:15:12.637315 23622 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:12.638777 23622 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 07:15:12.638885 23622 log.cc:1196] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260810 07:15:12.639222 23622 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324: Deleting consensus metadata
I20260810 07:15:12.639369 23622 master_runner.cc:315] Copying system tablet from 127.15.217.126:41187
I20260810 07:15:12.640035 23622 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324: tablet copy: Beginning tablet copy session from remote peer at address 127.15.217.126:41187
I20260810 07:15:12.646657 22957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.640482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51220 (local address 127.15.217.126:41187)
0810 07:15:12.640535 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:12.640537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:12.640679 (+ 142us) server_negotiation.cc:407] Connection header received
0810 07:15:12.640752 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:12.640753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:12.640764 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:12.640840 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:12.641186 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.641702 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.642508 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.642717 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.642807 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.643835 (+ 1028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:12.643837 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:12.643837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:12.643845 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:12.645424 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.645806 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.645807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.645808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.645877 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.646095 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.646096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.646097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.646216 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:12.646288 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:12.646470 (+ 182us) server_negotiation.cc:299] Negotiation successful
0810 07:15:12.646554 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 07:15:12.646690 23745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.640396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41187 (local address 127.0.0.1:51220)
0810 07:15:12.640592 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:12.640603 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:12.640693 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:12.640846 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:12.640847 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:12.641041 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:12.641096 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.641098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.641797 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.641798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.642412 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.642414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.642810 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.642811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.642883 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.643390 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:12.643396 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:12.643735 (+ 339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:12.645531 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:12.645532 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:12.645533 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:12.645726 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:12.645948 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:12.645949 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:12.645949 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:12.646012 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:12.646291 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:12.646293 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:12.646342 (+ 49us) client_negotiation.cc:769] Sending connection context
0810 07:15:12.646442 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 07:15:12.646566 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":104,"threads_started":1}
I20260810 07:15:12.647260 22933 tablet_copy_service.cc:140] P 29e10cc613d84f3e807c6339559f7930: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer c2d2d5a77d9f482f93cc231f8b7e5324 ({username='oryx', principal='oryx/127.15.217.124@KRBTEST.COM'} at 127.0.0.1:51220)
I20260810 07:15:12.647343 22933 tablet_copy_service.cc:161] P 29e10cc613d84f3e807c6339559f7930: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer c2d2d5a77d9f482f93cc231f8b7e5324 at {username='oryx', principal='oryx/127.15.217.124@KRBTEST.COM'} at 127.0.0.1:51220: session id = c2d2d5a77d9f482f93cc231f8b7e5324-00000000000000000000000000000000
I20260810 07:15:12.647843 22933 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 07:15:12.648351 23622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 07:15:12.649432 23622 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324: tablet copy: Starting download of 0 data blocks...
I20260810 07:15:12.649607 23622 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324: tablet copy: Starting download of 1 WAL segments...
I20260810 07:15:12.650337 23622 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 07:15:12.651098 22933 tablet_copy_service.cc:342] P 29e10cc613d84f3e807c6339559f7930: Request end of tablet copy session c2d2d5a77d9f482f93cc231f8b7e5324-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.15.217.124@KRBTEST.COM'} at 127.0.0.1:51220
I20260810 07:15:12.651153 22933 tablet_copy_service.cc:433] P 29e10cc613d84f3e807c6339559f7930: ending tablet copy session c2d2d5a77d9f482f93cc231f8b7e5324-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer c2d2d5a77d9f482f93cc231f8b7e5324
I20260810 07:15:12.653610 23622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:12.654846 23747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:12.654846 23746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:12.657416 23749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:12.657528 23622 server_base.cc:1034] running on GCE node
I20260810 07:15:12.657712 23622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:12.657873 23622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:12.658998 23622 hybrid_clock.cc:648] HybridClock initialized: now 1786346112658984 us; error 34 us; skew 500 ppm
Aug 10 07:15:12 dist-test-slave-10pc krb5kdc[22857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346112, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:12.660432 23622 init.cc:377] Logged in from keytab as oryx/127.15.217.124@KRBTEST.COM (short username oryx)
I20260810 07:15:12.660766 23622 webserver.cc:460] Webserver started at http://127.15.217.124:33341/ using document root <none> and password file <none>
I20260810 07:15:12.660888 23622 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:12.660923 23622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:12.661607 23622 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260810 07:15:12.662187 23756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:12.662356 23622 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 07:15:12.662405 23622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-2/wal
uuid: "c2d2d5a77d9f482f93cc231f8b7e5324"
format_stamp: "Formatted at 2026-08-10 07:15:12 on dist-test-slave-10pc"
server_key: "bb9189314ca46215356fb8b88297af52"
server_key_iv: "8fa66e873f993fcb4bfa4454aded3e28"
server_key_version: "encryptionkey@0"
I20260810 07:15:12.662469 23622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:12.680820 23622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:12.682798 23622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:12.686269 23622 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.124:41403
I20260810 07:15:12.686760 23622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/master-2/data/info.pb
I20260810 07:15:12.689361 23815 sys_catalog.cc:263] Verifying existing consensus state
I20260810 07:15:12.689985 23815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324: Bootstrap starting.
I20260810 07:15:12.691267 23814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.124:41403 every 8 connection(s)
I20260810 07:15:12.692273 23815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:15:12.692566 23815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324: Bootstrap complete.
I20260810 07:15:12.692792 23815 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: false } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: NON_VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: true } }
I20260810 07:15:12.692891 23815 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c2d2d5a77d9f482f93cc231f8b7e5324, State: Initialized, Role: LEARNER
I20260810 07:15:12.692938 23815 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: false } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: NON_VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: true } }
I20260810 07:15:12.693118 23815 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:12.693113 23818 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: false } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: NON_VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: true } } }
I20260810 07:15:12.693346 23818 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [sys.catalog]: This master's current role is: LEARNER
I20260810 07:15:12.694846 23830 catalog_manager.cc:1259] Loaded cluster ID: 213dbfc3d5314eb3b1002679033b6e06
I20260810 07:15:12.694936 23830 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324: loading cluster ID for follower catalog manager: success
I20260810 07:15:12.695978 23830 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324: acquiring CA information for follower catalog manager: success
I20260810 07:15:12.696280 23830 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 07:15:12.747542 23728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:12.747864 23728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:12.747905 23728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:12.749878 23728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:12.749963 23728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:12.750010 23728 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.65
I20260810 07:15:12.752419 23728 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:44299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.65.keytab
--principal=oryx/127.15.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.65:34201
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.217.65
--webserver_port=41719
--tserver_master_addrs=127.15.217.126:41187,127.15.217.125:44791,127.15.217.124:41403
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.65
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:12.752740 23728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:12.753046 23728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:12.759073 23840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:12.759104 23838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:12.759054 23836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:12.759570 23728 server_base.cc:1034] running on GCE node
I20260810 07:15:12.759873 23728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:12.760143 23728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:12.761325 23728 hybrid_clock.cc:648] HybridClock initialized: now 1786346112761304 us; error 38 us; skew 500 ppm
Aug 10 07:15:12 dist-test-slave-10pc krb5kdc[22857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346112, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:12.765480 23728 init.cc:377] Logged in from keytab as oryx/127.15.217.65@KRBTEST.COM (short username oryx)
I20260810 07:15:12.766088 23728 webserver.cc:460] Webserver started at http://127.15.217.65:41719/ using document root <none> and password file <none>
I20260810 07:15:12.766366 23728 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:12.766460 23728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:12.768792 23728 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 07:15:12.769690 23847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:12.769906 23728 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:12.770025 23728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-0/wal
uuid: "2a7624fba8924223967850a43cc913c2"
format_stamp: "Formatted at 2026-08-10 07:15:11 on dist-test-slave-10pc"
server_key: "a2f3ec1c5625a27ee5b497ceb110f772"
server_key_iv: "9b13edfdbe62fc386dde93af3a86c603"
server_key_version: "encryptionkey@0"
I20260810 07:15:12.770406 23728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:12.802636 23728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:12.805202 23728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:12.805418 23728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:12.805846 23728 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:12.806277 23728 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:12.806314 23728 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:12.806339 23728 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:12.806386 23728 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:12.823343 23728 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.65:34201
I20260810 07:15:12.823446 23266 heartbeater.cc:503] Master 127.15.217.126:41187 was elected leader, sending a full tablet report...
I20260810 07:15:12.823493 23971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.65:34201 every 8 connection(s)
I20260810 07:15:12.823851 23728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
I20260810 07:15:12.832298 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 23728
I20260810 07:15:12.832533 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 23128
Aug 10 07:15:12 dist-test-slave-10pc krb5kdc[22857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346112, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.124@KRBTEST.COM
Aug 10 07:15:12 dist-test-slave-10pc krb5kdc[22857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346112, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.125@KRBTEST.COM
I20260810 07:15:12.837970 23977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.824941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:47887 (local address 127.15.217.124:41403)
0810 07:15:12.825130 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:12.825133 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:12.825786 (+ 653us) server_negotiation.cc:407] Connection header received
0810 07:15:12.826992 (+ 1206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:12.826993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:12.827006 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:12.827119 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:12.828558 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.829321 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.830481 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.830746 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.830813 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.834331 (+ 3518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:12.834333 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:12.834333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:12.834341 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:12.836391 (+ 2050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.836910 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.836911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.836912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.836985 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.837249 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.837251 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.837251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.837381 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:12.837458 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:12.837747 (+ 289us) server_negotiation.cc:299] Negotiation successful
0810 07:15:12.837845 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":133,"threads_started":1}
I20260810 07:15:12.838269 23980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.824892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:41403 (local address 127.15.217.65:47887)
0810 07:15:12.825721 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:12.825733 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:12.826795 (+ 1062us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:12.827151 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:12.827152 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:12.827585 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:12.828071 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.828080 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.829418 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.829419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.830371 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.830374 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.830844 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.830845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.831090 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.831804 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:12.831814 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:12.834212 (+ 2398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:12.836519 (+ 2307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:12.836527 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:12.836529 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:12.836817 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:12.837056 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:12.837057 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:12.837057 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:12.837181 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:12.837448 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:12.837457 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:12.837609 (+ 152us) client_negotiation.cc:769] Sending connection context
0810 07:15:12.837724 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 07:15:12.837954 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":694,"mutex_wait_us":2,"thread_start_us":215,"threads_started":1}
I20260810 07:15:12.838868 23972 heartbeater.cc:347] Connected to a master server at 127.15.217.124:41403
I20260810 07:15:12.838989 23972 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:12.839229 23972 heartbeater.cc:511] Master 127.15.217.124:41403 requested a full tablet report, sending...
I20260810 07:15:12.839857 23775 ts_manager.cc:194] Registered new tserver with Master: 2a7624fba8924223967850a43cc913c2 (127.15.217.65:34201)
I20260810 07:15:12.839936 23619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.825156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:33513 (local address 127.15.217.125:44791)
0810 07:15:12.825204 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:12.825207 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:12.825496 (+ 289us) server_negotiation.cc:407] Connection header received
0810 07:15:12.826881 (+ 1385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:12.826882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:12.826893 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:12.826936 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:12.828176 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.828843 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.830690 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.830949 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.831029 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.835336 (+ 4307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:12.835338 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:12.835338 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:12.835347 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:12.837585 (+ 2238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.838799 (+ 1214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.838801 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.838801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.838896 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.839252 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.839253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.839253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.839403 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:12.839476 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:12.839670 (+ 194us) server_negotiation.cc:299] Negotiation successful
0810 07:15:12.839787 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 07:15:12.840366 23979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.824892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:44791 (local address 127.15.217.65:33513)
0810 07:15:12.825335 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:12.825371 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:12.826739 (+ 1368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:12.827032 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:12.827036 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:12.827585 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:12.828071 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.828080 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.829099 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.829100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.830600 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.830603 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.831032 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.831033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.831233 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.832013 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:12.832018 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:12.835225 (+ 3207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:12.837707 (+ 2482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:12.837708 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:12.837709 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:12.838706 (+ 997us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:12.839063 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:12.839064 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:12.839064 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:12.839167 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:12.839467 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:12.839468 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:12.839510 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 07:15:12.840072 (+ 562us) client_negotiation.cc:240] Negotiation successful
0810 07:15:12.840188 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":202,"threads_started":1}
I20260810 07:15:12.840916 23975 heartbeater.cc:347] Connected to a master server at 127.15.217.125:44791
I20260810 07:15:12.840986 23975 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:12.841085 23975 heartbeater.cc:511] Master 127.15.217.125:44791 requested a full tablet report, sending...
I20260810 07:15:12.841482 23560 ts_manager.cc:194] Registered new tserver with Master: 2a7624fba8924223967850a43cc913c2 (127.15.217.65:34201)
Aug 10 07:15:12 dist-test-slave-10pc krb5kdc[22857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346112, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM
I20260810 07:15:12.849996 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.66:46239
--local_ip_for_outbound_sockets=127.15.217.66
--tserver_master_addrs=127.15.217.126:41187
--webserver_port=35605
--webserver_interface=127.15.217.66
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.66.keytab
--principal=oryx/127.15.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:44299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.217.126:41187,127.15.217.125:44791
--tserver_master_addrs=127.15.217.126:41187,127.15.217.125:44791,127.15.217.124:41403 with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:15:12.850221 22957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.825491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:39833 (local address 127.15.217.126:41187)
0810 07:15:12.825546 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:12.825548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:12.828311 (+ 2763us) server_negotiation.cc:407] Connection header received
0810 07:15:12.828388 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:12.828389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:12.828401 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:12.828495 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:12.828870 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.829835 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.831148 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.831497 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.831569 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.847342 (+ 15773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:12.847343 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:12.847344 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:12.847353 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:12.848944 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.849316 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.849317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.849318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.849381 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:12.849654 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:12.849655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:12.849655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:12.849780 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:12.849863 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:12.850032 (+ 169us) server_negotiation.cc:299] Negotiation successful
0810 07:15:12.850122 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 07:15:12.850644 23978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.825393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41187 (local address 127.15.217.65:39833)
0810 07:15:12.828227 (+ 2834us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:12.828238 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:12.828318 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:12.828514 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:12.828515 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:12.828684 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:12.828776 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.828778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.829951 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.829952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.830980 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.830983 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.831592 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.831594 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.831743 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.845639 (+ 13896us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:12.845645 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:12.847220 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:12.849050 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:12.849051 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:12.849052 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:12.849231 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:12.849489 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:12.849491 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:12.849491 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:12.849581 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:12.849858 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:12.849859 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:12.849910 (+ 51us) client_negotiation.cc:769] Sending connection context
0810 07:15:12.850000 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 07:15:12.850188 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2816,"thread_start_us":120,"threads_started":1}
I20260810 07:15:12.851079 23974 heartbeater.cc:347] Connected to a master server at 127.15.217.126:41187
I20260810 07:15:12.851151 23974 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:12.851238 23974 heartbeater.cc:511] Master 127.15.217.126:41187 requested a full tablet report, sending...
I20260810 07:15:12.851595 22908 ts_manager.cc:194] Re-registered known tserver with Master: 2a7624fba8924223967850a43cc913c2 (127.15.217.65:34201)
I20260810 07:15:12.852000 22908 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.217.65@KRBTEST.COM'} at 127.15.217.65:39833
W20260810 07:15:12.944875 23983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:12.945226 23983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:12.945271 23983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:12.946568 23983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:12.946631 23983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:12.946674 23983 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.66
I20260810 07:15:12.948225 23983 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:44299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.66.keytab
--principal=oryx/127.15.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.66:46239
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.217.66
--webserver_port=35605
--tserver_master_addrs=127.15.217.126:41187,127.15.217.125:44791,127.15.217.124:41403
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.66
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:12.948552 23983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:12.948830 23983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:12.955241 23989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:12.955202 23988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:12.955513 23991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:12.955641 23983 server_base.cc:1034] running on GCE node
I20260810 07:15:12.955873 23983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:12.956166 23983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:12.957337 23983 hybrid_clock.cc:648] HybridClock initialized: now 1786346112957322 us; error 31 us; skew 500 ppm
Aug 10 07:15:12 dist-test-slave-10pc krb5kdc[22857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346112, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:12.960041 23983 init.cc:377] Logged in from keytab as oryx/127.15.217.66@KRBTEST.COM (short username oryx)
I20260810 07:15:12.960637 23983 webserver.cc:460] Webserver started at http://127.15.217.66:35605/ using document root <none> and password file <none>
I20260810 07:15:12.960882 23983 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:12.960971 23983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:12.962519 23983 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:15:12.963166 23998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:12.963398 23983 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:12.963506 23983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-1/wal
uuid: "87f6de6d8282454f9da57f1a17b89e6e"
format_stamp: "Formatted at 2026-08-10 07:15:11 on dist-test-slave-10pc"
server_key: "4877ab08011f1be5808742ebd46b4ac8"
server_key_iv: "79fc02607bba5559ccc79e6388bdcc25"
server_key_version: "encryptionkey@0"
I20260810 07:15:12.963938 23983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:12.980062 23983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:12.982635 23983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:12.982815 23983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:12.983086 23983 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:12.983433 23983 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:12.983490 23983 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:12.983533 23983 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:12.983562 23983 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:12.984988 23409 heartbeater.cc:503] Master 127.15.217.126:41187 was elected leader, sending a full tablet report...
I20260810 07:15:12.991971 23983 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.66:46239
I20260810 07:15:12.992039 24125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.66:46239 every 8 connection(s)
I20260810 07:15:12.992558 23983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
I20260810 07:15:12.997329 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 23983
I20260810 07:15:12.997592 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 23271
Aug 10 07:15:12 dist-test-slave-10pc krb5kdc[22857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346112, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.124@KRBTEST.COM
Aug 10 07:15:12 dist-test-slave-10pc krb5kdc[22857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346112, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.125@KRBTEST.COM
I20260810 07:15:13.005659 23619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.994289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:40991 (local address 127.15.217.125:44791)
0810 07:15:12.994332 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:12.994334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:12.994343 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:15:12.995427 (+ 1084us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:12.995429 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:12.995443 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:12.995496 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:12.996226 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.997058 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.998739 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.999019 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.999088 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:13.002036 (+ 2948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:13.002037 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:13.002037 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:13.002046 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:13.004180 (+ 2134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:13.004632 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:13.004633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:13.004634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:13.004702 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:13.004901 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:13.004902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:13.004902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:13.005077 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:13.005171 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:13.005462 (+ 291us) server_negotiation.cc:299] Negotiation successful
0810 07:15:13.005555 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 07:15:13.005883 23977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.993960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:43221 (local address 127.15.217.124:41403)
0810 07:15:12.994012 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:12.994015 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:12.994027 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:15:12.995453 (+ 1426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:12.995455 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:12.995471 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:12.995503 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:12.996376 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.997142 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.998561 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.998862 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.998904 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:13.002004 (+ 3100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:13.002006 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:13.002006 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:13.002017 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:13.003905 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:13.004537 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:13.004538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:13.004539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:13.004608 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:13.005205 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:13.005206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:13.005206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:13.005366 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:13.005439 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:13.005632 (+ 193us) server_negotiation.cc:299] Negotiation successful
0810 07:15:13.005740 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 07:15:13.005901 24133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.994055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:44791 (local address 127.15.217.66:40991)
0810 07:15:12.994233 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:12.994244 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:12.995320 (+ 1076us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:12.995590 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:12.995594 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:12.995886 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:12.996104 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.996108 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.997287 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.997288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.998657 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.998660 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.999106 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.999107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.999246 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.999908 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:12.999913 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:13.001909 (+ 1996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:13.004266 (+ 2357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:13.004267 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:13.004268 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:13.004550 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:13.004777 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:13.004778 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:13.004778 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:13.004831 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:13.005150 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:13.005157 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:13.005312 (+ 155us) client_negotiation.cc:769] Sending connection context
0810 07:15:13.005437 (+ 125us) client_negotiation.cc:240] Negotiation successful
0810 07:15:13.005626 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":112,"threads_started":1}
I20260810 07:15:13.006290 24131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.993448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:41403 (local address 127.15.217.66:43221)
0810 07:15:12.993728 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:12.993784 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:12.995307 (+ 1523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:12.995590 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:12.995595 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:12.996036 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:12.996260 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.996263 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.997273 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.997274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.998463 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.998466 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.999001 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:12.999002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:12.999156 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:12.999760 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:12.999772 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:13.001889 (+ 2117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:13.004004 (+ 2115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:13.004013 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:13.004016 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:13.004439 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:13.004673 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:13.004674 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:13.004675 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:13.004831 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:13.005470 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:13.005471 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:13.005519 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 07:15:13.005982 (+ 463us) client_negotiation.cc:240] Negotiation successful
0810 07:15:13.006108 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":91,"threads_started":1}
I20260810 07:15:13.006875 24128 heartbeater.cc:347] Connected to a master server at 127.15.217.125:44791
I20260810 07:15:13.006875 24126 heartbeater.cc:347] Connected to a master server at 127.15.217.124:41403
I20260810 07:15:13.007061 24128 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:13.007063 24126 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:13.007284 24128 heartbeater.cc:511] Master 127.15.217.125:44791 requested a full tablet report, sending...
I20260810 07:15:13.007287 24126 heartbeater.cc:511] Master 127.15.217.124:41403 requested a full tablet report, sending...
I20260810 07:15:13.007462 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.67:39201
--local_ip_for_outbound_sockets=127.15.217.67
--tserver_master_addrs=127.15.217.126:41187
--webserver_port=44561
--webserver_interface=127.15.217.67
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.67.keytab
--principal=oryx/127.15.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:44299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.217.126:41187,127.15.217.125:44791
--tserver_master_addrs=127.15.217.126:41187,127.15.217.125:44791,127.15.217.124:41403 with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:15:13.007889 23775 ts_manager.cc:194] Registered new tserver with Master: 87f6de6d8282454f9da57f1a17b89e6e (127.15.217.66:46239)
I20260810 07:15:13.007889 23560 ts_manager.cc:194] Registered new tserver with Master: 87f6de6d8282454f9da57f1a17b89e6e (127.15.217.66:46239)
Aug 10 07:15:13 dist-test-slave-10pc krb5kdc[22857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346112, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM
I20260810 07:15:13.014784 22957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.993885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:44077 (local address 127.15.217.126:41187)
0810 07:15:12.994015 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:12.994017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:12.994287 (+ 270us) server_negotiation.cc:407] Connection header received
0810 07:15:12.995453 (+ 1166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:12.995455 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:12.995469 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:12.995503 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:12.996429 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:12.996973 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.009217 (+ 12244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.009435 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.009526 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:13.011981 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:13.011983 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:13.011984 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:13.011992 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:13.013556 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:13.013942 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:13.013944 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:13.013944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:13.014013 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:13.014226 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:13.014227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:13.014227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:13.014348 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:13.014418 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:13.014586 (+ 168us) server_negotiation.cc:299] Negotiation successful
0810 07:15:13.014675 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260810 07:15:13.014808 24132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.993791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41187 (local address 127.15.217.66:44077)
0810 07:15:12.994206 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:12.994218 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:12.995342 (+ 1124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:12.995590 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:12.995595 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:12.996105 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:12.996324 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:12.996326 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.008530 (+ 12204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.008531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:13.009105 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:13.009106 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.009527 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.009528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:13.009604 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:13.010084 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:13.010090 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:13.011857 (+ 1767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:13.013662 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:13.013663 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:13.013664 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:13.013859 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:13.014084 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:13.014085 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:13.014085 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:13.014145 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:13.014425 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:13.014426 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:13.014470 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 07:15:13.014559 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 07:15:13.014675 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":124,"threads_started":1}
I20260810 07:15:13.015172 24127 heartbeater.cc:347] Connected to a master server at 127.15.217.126:41187
I20260810 07:15:13.015223 24127 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:13.015285 24127 heartbeater.cc:511] Master 127.15.217.126:41187 requested a full tablet report, sending...
I20260810 07:15:13.015601 22908 ts_manager.cc:194] Re-registered known tserver with Master: 87f6de6d8282454f9da57f1a17b89e6e (127.15.217.66:46239)
I20260810 07:15:13.015996 22908 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.217.66@KRBTEST.COM'} at 127.15.217.66:44077
W20260810 07:15:13.096911 24134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:13.097172 24134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:13.097216 24134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:13.098425 24134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:13.098496 24134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:13.098541 24134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.67
I20260810 07:15:13.100142 24134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:44299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.67.keytab
--principal=oryx/127.15.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.67:39201
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.217.67
--webserver_port=44561
--tserver_master_addrs=127.15.217.126:41187,127.15.217.125:44791,127.15.217.124:41403
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.67
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:13.100435 24134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:13.100720 24134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:13.106446 24140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:13.106456 24142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:13.106456 24139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:13.106729 24134 server_base.cc:1034] running on GCE node
I20260810 07:15:13.106992 24134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:13.107272 24134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:13.108441 24134 hybrid_clock.cc:648] HybridClock initialized: now 1786346113108437 us; error 39 us; skew 500 ppm
Aug 10 07:15:13 dist-test-slave-10pc krb5kdc[22857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346113, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:13.111148 24134 init.cc:377] Logged in from keytab as oryx/127.15.217.67@KRBTEST.COM (short username oryx)
I20260810 07:15:13.111604 24134 webserver.cc:460] Webserver started at http://127.15.217.67:44561/ using document root <none> and password file <none>
I20260810 07:15:13.111852 24134 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:13.111938 24134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:13.113461 24134 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260810 07:15:13.114095 24149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:13.114307 24134 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:13.114428 24134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-2/wal
uuid: "ceed6041617a45b6ab60d5fb2064d12d"
format_stamp: "Formatted at 2026-08-10 07:15:11 on dist-test-slave-10pc"
server_key: "ce68b1ef4a6d2af284f63425f5748de5"
server_key_iv: "0a1ee665952f346aee60aa8a4a08288f"
server_key_version: "encryptionkey@0"
I20260810 07:15:13.114835 24134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:13.132860 24134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:13.135061 24134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:13.135247 24134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:13.135517 24134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:13.135872 24134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:13.135931 24134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:13.135978 24134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:13.136013 24134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:13.142395 24134 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.67:39201
I20260810 07:15:13.142442 24276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.67:39201 every 8 connection(s)
I20260810 07:15:13.142822 24134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
I20260810 07:15:13.144102 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 24134
Aug 10 07:15:13 dist-test-slave-10pc krb5kdc[22857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346113, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.124@KRBTEST.COM
Aug 10 07:15:13 dist-test-slave-10pc krb5kdc[22857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346113, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.125@KRBTEST.COM
Aug 10 07:15:13 dist-test-slave-10pc krb5kdc[22857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346113, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM
I20260810 07:15:13.157676 24283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:13.143617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:44791 (local address 127.15.217.67:34891)
0810 07:15:13.144127 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:13.144138 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:13.145438 (+ 1300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:13.145809 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:13.145814 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:13.146412 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:13.146745 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:13.146752 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.147790 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.147791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:13.149345 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:13.149349 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.149882 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.149883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:13.150045 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:13.151205 (+ 1160us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:13.151211 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:13.153504 (+ 2293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:13.155635 (+ 2131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:13.155645 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:13.155649 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:13.156000 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:13.156220 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:13.156221 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:13.156222 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:13.156377 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:13.156764 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:13.156773 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:13.156967 (+ 194us) client_negotiation.cc:769] Sending connection context
0810 07:15:13.157074 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 07:15:13.157292 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"mutex_wait_us":118,"thread_start_us":146,"threads_started":1}
I20260810 07:15:13.158134 23619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:13.143612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:34891 (local address 127.15.217.125:44791)
0810 07:15:13.143666 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:13.143668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:13.144215 (+ 547us) server_negotiation.cc:407] Connection header received
0810 07:15:13.145560 (+ 1345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:13.145561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:13.145577 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:13.145633 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:13.146865 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.147645 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.149451 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.149764 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.149821 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:13.153620 (+ 3799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:13.153621 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:13.153622 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:13.153629 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:13.155527 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:13.156085 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:13.156086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:13.156087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:13.156149 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:13.156502 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:13.156503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:13.156504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:13.156653 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:13.156732 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:13.157951 (+ 1219us) server_negotiation.cc:299] Negotiation successful
0810 07:15:13.158052 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 07:15:13.159303 24284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:13.143987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:41403 (local address 127.15.217.67:44535)
0810 07:15:13.144273 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:13.144281 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:13.145594 (+ 1313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:13.145809 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:13.145814 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:13.146416 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:13.146745 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:13.146752 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.147790 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.147791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:13.149349 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:13.149350 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.149880 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.149881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:13.150052 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:13.150977 (+ 925us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:13.150982 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:13.153881 (+ 2899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:13.157048 (+ 3167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:13.157049 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:13.157050 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:13.157346 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:13.157558 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:13.157559 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:13.157559 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:13.157636 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:13.158484 (+ 848us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:13.158485 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:13.158537 (+ 52us) client_negotiation.cc:769] Sending connection context
0810 07:15:13.158691 (+ 154us) client_negotiation.cc:240] Negotiation successful
0810 07:15:13.158813 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":86,"thread_start_us":62,"threads_started":1}
I20260810 07:15:13.159678 22957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:13.143622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:39129 (local address 127.15.217.126:41187)
0810 07:15:13.143673 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:13.143675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:13.144003 (+ 328us) server_negotiation.cc:407] Connection header received
0810 07:15:13.145889 (+ 1886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:13.145891 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:13.145904 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:13.145938 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:13.146875 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.147692 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.149446 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.149729 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.149772 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:13.155354 (+ 5582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:13.155355 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:13.155355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:13.155363 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:13.158044 (+ 2681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:13.158804 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:13.158805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:13.158806 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:13.158870 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:13.159090 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:13.159091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:13.159092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:13.159227 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:13.159305 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:13.159485 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 07:15:13.159584 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 07:15:13.159874 23977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:13.144132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:44535 (local address 127.15.217.124:41403)
0810 07:15:13.144192 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:13.144195 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:13.144343 (+ 148us) server_negotiation.cc:407] Connection header received
0810 07:15:13.145682 (+ 1339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:13.145684 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:13.145695 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:13.145753 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:13.146866 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.147645 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.149451 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.149759 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.149821 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:13.154004 (+ 4183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:13.154005 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:13.154005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:13.154013 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:13.156961 (+ 2948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:13.157432 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:13.157433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:13.157433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:13.157494 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:13.157723 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:13.157725 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:13.157725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:13.157868 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:13.157928 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:13.159659 (+ 1731us) server_negotiation.cc:299] Negotiation successful
0810 07:15:13.159754 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 07:15:13.159355 24280 heartbeater.cc:347] Connected to a master server at 127.15.217.125:44791
I20260810 07:15:13.160606 24280 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:13.160604 24278 heartbeater.cc:347] Connected to a master server at 127.15.217.124:41403
I20260810 07:15:13.160686 24278 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:13.159765 24282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:13.143617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41187 (local address 127.15.217.67:39129)
0810 07:15:13.143850 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:13.143897 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:13.145790 (+ 1893us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:13.146020 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:13.146021 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:13.146398 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:13.146746 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:13.146755 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.147799 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.147800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:13.149324 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:13.149327 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.149863 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.149864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:13.150038 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:13.150731 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:13.150742 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:13.155235 (+ 4493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:13.158461 (+ 3226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:13.158462 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:13.158462 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:13.158710 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:13.158945 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:13.158946 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:13.158946 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:13.159011 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:13.159312 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:13.159313 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:13.159357 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 07:15:13.159547 (+ 190us) client_negotiation.cc:240] Negotiation successful
0810 07:15:13.159651 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":162,"thread_start_us":148,"threads_started":1}
I20260810 07:15:13.160883 24278 heartbeater.cc:511] Master 127.15.217.124:41403 requested a full tablet report, sending...
I20260810 07:15:13.161033 24294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:13.145440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34876 (local address 127.15.217.124:41403)
0810 07:15:13.145739 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:13.145741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:13.146125 (+ 384us) server_negotiation.cc:407] Connection header received
0810 07:15:13.148675 (+ 2550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:13.148676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:13.148688 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:13.148727 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:13.150251 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.150818 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.152889 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.153220 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.153263 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:13.155826 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:13.155827 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:13.155828 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:13.155835 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:13.157975 (+ 2140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:13.158424 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:13.158425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:13.158425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:13.158490 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:13.160230 (+ 1740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:13.160231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:13.160231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:13.160395 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:13.160501 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:13.160768 (+ 267us) server_negotiation.cc:299] Negotiation successful
0810 07:15:13.160910 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":44,"threads_started":1}
I20260810 07:15:13.161284 24279 heartbeater.cc:347] Connected to a master server at 127.15.217.126:41187
I20260810 07:15:13.161335 24279 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:13.161406 24279 heartbeater.cc:511] Master 127.15.217.126:41187 requested a full tablet report, sending...
I20260810 07:15:13.161752 23774 ts_manager.cc:194] Registered new tserver with Master: ceed6041617a45b6ab60d5fb2064d12d (127.15.217.67:39201)
I20260810 07:15:13.162112 22908 ts_manager.cc:194] Re-registered known tserver with Master: ceed6041617a45b6ab60d5fb2064d12d (127.15.217.67:39201)
I20260810 07:15:13.162225 24291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:13.145225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51222 (local address 127.15.217.126:41187)
0810 07:15:13.145938 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:13.145941 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:13.145948 (+ 7us) server_negotiation.cc:407] Connection header received
0810 07:15:13.145965 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:13.145966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:13.145977 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:13.147724 (+ 1747us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:13.147763 (+ 39us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.148495 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.154017 (+ 5522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.154351 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.154696 (+ 345us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:13.156125 (+ 1429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:13.156127 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:13.156127 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:13.156134 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:13.159885 (+ 3751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:13.160642 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:13.160643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:13.160644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:13.161184 (+ 540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:13.161424 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:13.161425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:13.161426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:13.161580 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:13.161953 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:13.161999 (+ 46us) server_negotiation.cc:299] Negotiation successful
0810 07:15:13.162092 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":691,"thread_start_us":77,"threads_started":1}
I20260810 07:15:13.162643 22908 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.217.67@KRBTEST.COM'} at 127.15.217.67:39129
I20260810 07:15:13.160883 24280 heartbeater.cc:511] Master 127.15.217.125:44791 requested a full tablet report, sending...
I20260810 07:15:13.163784 23560 ts_manager.cc:194] Registered new tserver with Master: ceed6041617a45b6ab60d5fb2064d12d (127.15.217.67:39201)
I20260810 07:15:13.163796 24292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:13.145379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48304 (local address 127.15.217.125:44791)
0810 07:15:13.146180 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:13.146182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:13.146189 (+ 7us) server_negotiation.cc:407] Connection header received
0810 07:15:13.146210 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:13.146211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:13.146221 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:13.146284 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:13.149420 (+ 3136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.150131 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.152787 (+ 2656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.154026 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.158330 (+ 4304us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:13.159901 (+ 1571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:13.159903 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:13.159903 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:13.159910 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:13.162174 (+ 2264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:13.162602 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:13.162603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:13.162604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:13.162667 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:13.162888 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:13.162889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:13.162889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:13.163095 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:13.163202 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:13.163611 (+ 409us) server_negotiation.cc:299] Negotiation successful
0810 07:15:13.163707 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":780,"thread_start_us":57,"threads_started":1}
I20260810 07:15:13.164918 22908 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51222:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:15:13.166527 22908 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:15:13.175612 24309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:13.171146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:39201 (local address 127.0.0.1:42584)
0810 07:15:13.171715 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:13.171725 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:13.171777 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:13.172045 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:13.172046 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:13.172048 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:13.172080 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:13.172081 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.173490 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.173491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:13.174337 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:13.174341 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.175323 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.175324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:13.175384 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:13.175389 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:15:13.175449 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 07:15:13.175526 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":153,"threads_started":1}
I20260810 07:15:13.175694 24308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:13.171269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42584 (local address 127.15.217.67:39201)
0810 07:15:13.171905 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:13.171908 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:13.171918 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:15:13.171957 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:13.171959 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:13.171974 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:13.172047 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:13.172141 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.173354 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.174420 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.175227 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.175324 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:13.175502 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:13.175535 (+ 33us) server_negotiation.cc:299] Negotiation successful
0810 07:15:13.175578 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":161,"threads_started":1}
I20260810 07:15:13.175858 24307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:13.170899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:46239 (local address 127.0.0.1:37898)
0810 07:15:13.171496 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:13.171516 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:13.171612 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:13.171934 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:13.171935 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:13.171937 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:13.171996 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:13.171998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.173140 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.173141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:13.174437 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:13.174440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.175589 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.175590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:13.175655 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:13.175661 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 07:15:13.175708 (+ 47us) client_negotiation.cc:240] Negotiation successful
0810 07:15:13.175750 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":594,"threads_started":1}
I20260810 07:15:13.176281 24303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:13.170789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:34201 (local address 127.0.0.1:34474)
0810 07:15:13.171482 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:13.171496 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:13.171556 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:13.172101 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:13.172102 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:13.172104 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:13.172158 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:13.172161 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.173545 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.173547 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:13.174721 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:13.174724 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.176005 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.176006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:13.176068 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:13.176073 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:15:13.176137 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 07:15:13.176217 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":242,"threads_started":1}
I20260810 07:15:13.176476 24306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:13.171077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37898 (local address 127.15.217.66:46239)
0810 07:15:13.171356 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:13.171362 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:13.171578 (+ 216us) server_negotiation.cc:407] Connection header received
0810 07:15:13.171802 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:13.171805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:13.171831 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:13.171924 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:13.172076 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.173044 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.174633 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.175477 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.175990 (+ 513us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:13.176218 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:13.176252 (+ 34us) server_negotiation.cc:299] Negotiation successful
0810 07:15:13.176302 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":180,"threads_started":1}
I20260810 07:15:13.176661 24199 tablet_service.cc:1467] Processing CreateTablet for tablet a3d4d72c0568419b8b94becc36f0e3f0 (DEFAULT_TABLE table=test-table [id=c3c7c50c5bac40439ea7abb656437ed9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:13.176826 24305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:13.170938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34474 (local address 127.15.217.65:34201)
0810 07:15:13.171804 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:13.171809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:13.171821 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:15:13.171900 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:13.171903 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:13.171921 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:13.172017 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:13.172243 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.173457 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.174810 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.175675 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.176377 (+ 702us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:13.176585 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:13.176621 (+ 36us) server_negotiation.cc:299] Negotiation successful
0810 07:15:13.176673 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":787,"thread_start_us":92,"threads_started":1}
I20260810 07:15:13.177138 24199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3d4d72c0568419b8b94becc36f0e3f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:13.177276 24051 tablet_service.cc:1467] Processing CreateTablet for tablet a3d4d72c0568419b8b94becc36f0e3f0 (DEFAULT_TABLE table=test-table [id=c3c7c50c5bac40439ea7abb656437ed9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:13.177708 24051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3d4d72c0568419b8b94becc36f0e3f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:13.181617 24310 tablet_bootstrap.cc:492] T a3d4d72c0568419b8b94becc36f0e3f0 P ceed6041617a45b6ab60d5fb2064d12d: Bootstrap starting.
I20260810 07:15:13.182581 24311 tablet_bootstrap.cc:492] T a3d4d72c0568419b8b94becc36f0e3f0 P 87f6de6d8282454f9da57f1a17b89e6e: Bootstrap starting.
I20260810 07:15:13.182561 23899 tablet_service.cc:1467] Processing CreateTablet for tablet a3d4d72c0568419b8b94becc36f0e3f0 (DEFAULT_TABLE table=test-table [id=c3c7c50c5bac40439ea7abb656437ed9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:13.182971 24310 tablet_bootstrap.cc:654] T a3d4d72c0568419b8b94becc36f0e3f0 P ceed6041617a45b6ab60d5fb2064d12d: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:13.182984 23899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3d4d72c0568419b8b94becc36f0e3f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:13.183924 24310 log.cc:824] T a3d4d72c0568419b8b94becc36f0e3f0 P ceed6041617a45b6ab60d5fb2064d12d: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:13.184059 24311 tablet_bootstrap.cc:654] T a3d4d72c0568419b8b94becc36f0e3f0 P 87f6de6d8282454f9da57f1a17b89e6e: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:13.184947 24311 log.cc:824] T a3d4d72c0568419b8b94becc36f0e3f0 P 87f6de6d8282454f9da57f1a17b89e6e: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:13.185262 24310 tablet_bootstrap.cc:492] T a3d4d72c0568419b8b94becc36f0e3f0 P ceed6041617a45b6ab60d5fb2064d12d: No bootstrap required, opened a new log
I20260810 07:15:13.185343 24310 ts_tablet_manager.cc:1397] T a3d4d72c0568419b8b94becc36f0e3f0 P ceed6041617a45b6ab60d5fb2064d12d: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260810 07:15:13.186226 24311 tablet_bootstrap.cc:492] T a3d4d72c0568419b8b94becc36f0e3f0 P 87f6de6d8282454f9da57f1a17b89e6e: No bootstrap required, opened a new log
I20260810 07:15:13.186301 24311 ts_tablet_manager.cc:1397] T a3d4d72c0568419b8b94becc36f0e3f0 P 87f6de6d8282454f9da57f1a17b89e6e: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260810 07:15:13.188417 24314 tablet_bootstrap.cc:492] T a3d4d72c0568419b8b94becc36f0e3f0 P 2a7624fba8924223967850a43cc913c2: Bootstrap starting.
I20260810 07:15:13.190115 24314 tablet_bootstrap.cc:654] T a3d4d72c0568419b8b94becc36f0e3f0 P 2a7624fba8924223967850a43cc913c2: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:13.190102 24310 raft_consensus.cc:348] T a3d4d72c0568419b8b94becc36f0e3f0 P ceed6041617a45b6ab60d5fb2064d12d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a7624fba8924223967850a43cc913c2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34201 } } peers { permanent_uuid: "ceed6041617a45b6ab60d5fb2064d12d" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39201 } } peers { permanent_uuid: "87f6de6d8282454f9da57f1a17b89e6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46239 } }
I20260810 07:15:13.190380 24310 raft_consensus.cc:374] T a3d4d72c0568419b8b94becc36f0e3f0 P ceed6041617a45b6ab60d5fb2064d12d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:13.190438 24310 raft_consensus.cc:729] T a3d4d72c0568419b8b94becc36f0e3f0 P ceed6041617a45b6ab60d5fb2064d12d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceed6041617a45b6ab60d5fb2064d12d, State: Initialized, Role: FOLLOWER
I20260810 07:15:13.190659 24310 consensus_queue.cc:260] T a3d4d72c0568419b8b94becc36f0e3f0 P ceed6041617a45b6ab60d5fb2064d12d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a7624fba8924223967850a43cc913c2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34201 } } peers { permanent_uuid: "ceed6041617a45b6ab60d5fb2064d12d" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39201 } } peers { permanent_uuid: "87f6de6d8282454f9da57f1a17b89e6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46239 } }
I20260810 07:15:13.191077 24310 ts_tablet_manager.cc:1428] T a3d4d72c0568419b8b94becc36f0e3f0 P ceed6041617a45b6ab60d5fb2064d12d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 07:15:13.191236 24279 heartbeater.cc:503] Master 127.15.217.126:41187 was elected leader, sending a full tablet report...
I20260810 07:15:13.191881 24314 log.cc:824] T a3d4d72c0568419b8b94becc36f0e3f0 P 2a7624fba8924223967850a43cc913c2: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:13.194640 24311 raft_consensus.cc:348] T a3d4d72c0568419b8b94becc36f0e3f0 P 87f6de6d8282454f9da57f1a17b89e6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a7624fba8924223967850a43cc913c2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34201 } } peers { permanent_uuid: "ceed6041617a45b6ab60d5fb2064d12d" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39201 } } peers { permanent_uuid: "87f6de6d8282454f9da57f1a17b89e6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46239 } }
I20260810 07:15:13.194844 24311 raft_consensus.cc:374] T a3d4d72c0568419b8b94becc36f0e3f0 P 87f6de6d8282454f9da57f1a17b89e6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:13.194891 24311 raft_consensus.cc:729] T a3d4d72c0568419b8b94becc36f0e3f0 P 87f6de6d8282454f9da57f1a17b89e6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87f6de6d8282454f9da57f1a17b89e6e, State: Initialized, Role: FOLLOWER
I20260810 07:15:13.195003 24314 tablet_bootstrap.cc:492] T a3d4d72c0568419b8b94becc36f0e3f0 P 2a7624fba8924223967850a43cc913c2: No bootstrap required, opened a new log
I20260810 07:15:13.195030 24311 consensus_queue.cc:260] T a3d4d72c0568419b8b94becc36f0e3f0 P 87f6de6d8282454f9da57f1a17b89e6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a7624fba8924223967850a43cc913c2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34201 } } peers { permanent_uuid: "ceed6041617a45b6ab60d5fb2064d12d" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39201 } } peers { permanent_uuid: "87f6de6d8282454f9da57f1a17b89e6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46239 } }
I20260810 07:15:13.195101 24314 ts_tablet_manager.cc:1397] T a3d4d72c0568419b8b94becc36f0e3f0 P 2a7624fba8924223967850a43cc913c2: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.003s
I20260810 07:15:13.195479 24311 ts_tablet_manager.cc:1428] T a3d4d72c0568419b8b94becc36f0e3f0 P 87f6de6d8282454f9da57f1a17b89e6e: Time spent starting tablet: real 0.009s user 0.005s sys 0.002s
I20260810 07:15:13.195612 24127 heartbeater.cc:503] Master 127.15.217.126:41187 was elected leader, sending a full tablet report...
I20260810 07:15:13.199467 24314 raft_consensus.cc:348] T a3d4d72c0568419b8b94becc36f0e3f0 P 2a7624fba8924223967850a43cc913c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a7624fba8924223967850a43cc913c2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34201 } } peers { permanent_uuid: "ceed6041617a45b6ab60d5fb2064d12d" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39201 } } peers { permanent_uuid: "87f6de6d8282454f9da57f1a17b89e6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46239 } }
I20260810 07:15:13.199761 24314 raft_consensus.cc:374] T a3d4d72c0568419b8b94becc36f0e3f0 P 2a7624fba8924223967850a43cc913c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:13.199846 24314 raft_consensus.cc:729] T a3d4d72c0568419b8b94becc36f0e3f0 P 2a7624fba8924223967850a43cc913c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a7624fba8924223967850a43cc913c2, State: Initialized, Role: FOLLOWER
I20260810 07:15:13.199997 24314 consensus_queue.cc:260] T a3d4d72c0568419b8b94becc36f0e3f0 P 2a7624fba8924223967850a43cc913c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a7624fba8924223967850a43cc913c2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34201 } } peers { permanent_uuid: "ceed6041617a45b6ab60d5fb2064d12d" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39201 } } peers { permanent_uuid: "87f6de6d8282454f9da57f1a17b89e6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46239 } }
I20260810 07:15:13.200420 24314 ts_tablet_manager.cc:1428] T a3d4d72c0568419b8b94becc36f0e3f0 P 2a7624fba8924223967850a43cc913c2: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260810 07:15:13.200614 23974 heartbeater.cc:503] Master 127.15.217.126:41187 was elected leader, sending a full tablet report...
I20260810 07:15:13.206646 24303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:13.203801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:41403 (local address 127.0.0.1:34888)
0810 07:15:13.203869 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:13.203881 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:13.203927 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:13.204155 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:13.204156 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:13.204158 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:13.204220 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:13.204222 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.205005 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.205006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:13.205766 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:13.205768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.206381 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.206382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:13.206449 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:13.206453 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:15:13.206510 (+ 57us) client_negotiation.cc:240] Negotiation successful
0810 07:15:13.206579 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 07:15:13.206646 24294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:13.203878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34888 (local address 127.15.217.124:41403)
0810 07:15:13.203933 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:13.203936 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:13.203962 (+ 26us) server_negotiation.cc:407] Connection header received
0810 07:15:13.203992 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:13.203993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:13.204018 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:13.204146 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:13.204314 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.204897 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.205864 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.206290 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.206383 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:13.206474 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:13.206520 (+ 46us) server_negotiation.cc:299] Negotiation successful
0810 07:15:13.206567 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 07:15:13.207165 23785 raft_consensus.cc:1203] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260810 07:15:13.207652 24319 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29e10cc613d84f3e807c6339559f7930. Latest consensus state: current_term: 1 leader_uuid: "29e10cc613d84f3e807c6339559f7930" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: false } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: NON_VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: true } } }
I20260810 07:15:13.207864 24319 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [sys.catalog]: This master's current role is: LEARNER
W20260810 07:15:13.243794 24130 tablet.cc:2369] T a3d4d72c0568419b8b94becc36f0e3f0 P 87f6de6d8282454f9da57f1a17b89e6e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:15:13.321398 24317 raft_consensus.cc:482] T a3d4d72c0568419b8b94becc36f0e3f0 P 87f6de6d8282454f9da57f1a17b89e6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:15:13.321560 24317 raft_consensus.cc:504] T a3d4d72c0568419b8b94becc36f0e3f0 P 87f6de6d8282454f9da57f1a17b89e6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a7624fba8924223967850a43cc913c2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34201 } } peers { permanent_uuid: "ceed6041617a45b6ab60d5fb2064d12d" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39201 } } peers { permanent_uuid: "87f6de6d8282454f9da57f1a17b89e6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46239 } }
I20260810 07:15:13.321935 24317 leader_election.cc:288] T a3d4d72c0568419b8b94becc36f0e3f0 P 87f6de6d8282454f9da57f1a17b89e6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a7624fba8924223967850a43cc913c2 (127.15.217.65:34201), ceed6041617a45b6ab60d5fb2064d12d (127.15.217.67:39201)
W20260810 07:15:13.325235 23976 tablet.cc:2369] T a3d4d72c0568419b8b94becc36f0e3f0 P 2a7624fba8924223967850a43cc913c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:15:13.325541 24132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:13.322216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:34201 (local address 127.15.217.66:43797)
0810 07:15:13.322287 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:13.322304 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:13.322381 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:13.322637 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:13.322638 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:13.322640 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:13.322824 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:13.322826 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.323586 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.323587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:13.324492 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:13.324495 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.325177 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.325178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:13.325339 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:13.325344 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:15:13.325402 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 07:15:13.325460 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 07:15:13.325604 24305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:13.322310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:43797 (local address 127.15.217.65:34201)
0810 07:15:13.322374 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:13.322376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:13.322388 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:15:13.322514 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:13.322515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:13.322526 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:13.322607 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:13.322935 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.323494 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.324589 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.325088 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.325177 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:13.325188 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:13.325488 (+ 300us) server_negotiation.cc:299] Negotiation successful
0810 07:15:13.325532 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 07:15:13.325975 24131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:13.322369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:39201 (local address 127.15.217.66:46061)
0810 07:15:13.322446 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:13.322458 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:13.322601 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:13.322779 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:13.322780 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:13.322782 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:13.322977 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:13.322979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.323711 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.323712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:13.324784 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:13.324786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.325575 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.325576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:13.325784 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:13.325789 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:15:13.325851 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 07:15:13.325898 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":11}
I20260810 07:15:13.325975 24308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:13.322445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:46061 (local address 127.15.217.67:39201)
0810 07:15:13.322500 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:13.322502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:13.322588 (+ 86us) server_negotiation.cc:407] Connection header received
0810 07:15:13.322675 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:13.322676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:13.322688 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:13.322749 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:13.323069 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.323557 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.324886 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.325448 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.325530 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:13.325540 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:13.325861 (+ 321us) server_negotiation.cc:299] Negotiation successful
0810 07:15:13.325912 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 07:15:13.326182 23920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3d4d72c0568419b8b94becc36f0e3f0" candidate_uuid: "87f6de6d8282454f9da57f1a17b89e6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a7624fba8924223967850a43cc913c2" is_pre_election: true
I20260810 07:15:13.326381 23920 raft_consensus.cc:2454] T a3d4d72c0568419b8b94becc36f0e3f0 P 2a7624fba8924223967850a43cc913c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87f6de6d8282454f9da57f1a17b89e6e in term 0.
I20260810 07:15:13.326620 24222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3d4d72c0568419b8b94becc36f0e3f0" candidate_uuid: "87f6de6d8282454f9da57f1a17b89e6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceed6041617a45b6ab60d5fb2064d12d" is_pre_election: true
I20260810 07:15:13.326689 24003 leader_election.cc:302] T a3d4d72c0568419b8b94becc36f0e3f0 P 87f6de6d8282454f9da57f1a17b89e6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a7624fba8924223967850a43cc913c2, 87f6de6d8282454f9da57f1a17b89e6e; no voters:
I20260810 07:15:13.326817 24222 raft_consensus.cc:2454] T a3d4d72c0568419b8b94becc36f0e3f0 P ceed6041617a45b6ab60d5fb2064d12d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87f6de6d8282454f9da57f1a17b89e6e in term 0.
I20260810 07:15:13.326836 24317 raft_consensus.cc:2781] T a3d4d72c0568419b8b94becc36f0e3f0 P 87f6de6d8282454f9da57f1a17b89e6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:15:13.326902 24317 raft_consensus.cc:482] T a3d4d72c0568419b8b94becc36f0e3f0 P 87f6de6d8282454f9da57f1a17b89e6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:15:13.326937 24317 raft_consensus.cc:3037] T a3d4d72c0568419b8b94becc36f0e3f0 P 87f6de6d8282454f9da57f1a17b89e6e [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:13.328099 24317 raft_consensus.cc:504] T a3d4d72c0568419b8b94becc36f0e3f0 P 87f6de6d8282454f9da57f1a17b89e6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a7624fba8924223967850a43cc913c2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34201 } } peers { permanent_uuid: "ceed6041617a45b6ab60d5fb2064d12d" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39201 } } peers { permanent_uuid: "87f6de6d8282454f9da57f1a17b89e6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46239 } }
I20260810 07:15:13.328261 24317 leader_election.cc:288] T a3d4d72c0568419b8b94becc36f0e3f0 P 87f6de6d8282454f9da57f1a17b89e6e [CANDIDATE]: Term 1 election: Requested vote from peers 2a7624fba8924223967850a43cc913c2 (127.15.217.65:34201), ceed6041617a45b6ab60d5fb2064d12d (127.15.217.67:39201)
I20260810 07:15:13.328436 23920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3d4d72c0568419b8b94becc36f0e3f0" candidate_uuid: "87f6de6d8282454f9da57f1a17b89e6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a7624fba8924223967850a43cc913c2"
I20260810 07:15:13.328521 24222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3d4d72c0568419b8b94becc36f0e3f0" candidate_uuid: "87f6de6d8282454f9da57f1a17b89e6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceed6041617a45b6ab60d5fb2064d12d"
I20260810 07:15:13.328590 23920 raft_consensus.cc:3037] T a3d4d72c0568419b8b94becc36f0e3f0 P 2a7624fba8924223967850a43cc913c2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:13.328625 24222 raft_consensus.cc:3037] T a3d4d72c0568419b8b94becc36f0e3f0 P ceed6041617a45b6ab60d5fb2064d12d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:13.329717 23920 raft_consensus.cc:2454] T a3d4d72c0568419b8b94becc36f0e3f0 P 2a7624fba8924223967850a43cc913c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87f6de6d8282454f9da57f1a17b89e6e in term 1.
I20260810 07:15:13.329874 24222 raft_consensus.cc:2454] T a3d4d72c0568419b8b94becc36f0e3f0 P ceed6041617a45b6ab60d5fb2064d12d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87f6de6d8282454f9da57f1a17b89e6e in term 1.
I20260810 07:15:13.329902 24003 leader_election.cc:302] T a3d4d72c0568419b8b94becc36f0e3f0 P 87f6de6d8282454f9da57f1a17b89e6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a7624fba8924223967850a43cc913c2, 87f6de6d8282454f9da57f1a17b89e6e; no voters:
I20260810 07:15:13.330011 24317 raft_consensus.cc:2781] T a3d4d72c0568419b8b94becc36f0e3f0 P 87f6de6d8282454f9da57f1a17b89e6e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:13.330271 24317 raft_consensus.cc:686] T a3d4d72c0568419b8b94becc36f0e3f0 P 87f6de6d8282454f9da57f1a17b89e6e [term 1 LEADER]: Becoming Leader. State: Replica: 87f6de6d8282454f9da57f1a17b89e6e, State: Running, Role: LEADER
I20260810 07:15:13.330402 24317 consensus_queue.cc:237] T a3d4d72c0568419b8b94becc36f0e3f0 P 87f6de6d8282454f9da57f1a17b89e6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a7624fba8924223967850a43cc913c2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34201 } } peers { permanent_uuid: "ceed6041617a45b6ab60d5fb2064d12d" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39201 } } peers { permanent_uuid: "87f6de6d8282454f9da57f1a17b89e6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46239 } }
I20260810 07:15:13.331423 22907 catalog_manager.cc:5495] T a3d4d72c0568419b8b94becc36f0e3f0 P 87f6de6d8282454f9da57f1a17b89e6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 87f6de6d8282454f9da57f1a17b89e6e (127.15.217.66). New cstate: current_term: 1 leader_uuid: "87f6de6d8282454f9da57f1a17b89e6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a7624fba8924223967850a43cc913c2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34201 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceed6041617a45b6ab60d5fb2064d12d" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39201 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87f6de6d8282454f9da57f1a17b89e6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46239 } health_report { overall_health: HEALTHY } } }
I20260810 07:15:13.332243 24297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "29e10cc613d84f3e807c6339559f7930" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: false } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: NON_VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: true } } }
I20260810 07:15:13.332355 24297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:13.333469 24326 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930: attempting to promote NON_VOTER c2d2d5a77d9f482f93cc231f8b7e5324 to VOTER
I20260810 07:15:13.333599 24326 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: false } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: false } }
I20260810 07:15:13.334131 23785 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [term 1 LEARNER]: Refusing update from remote peer 29e10cc613d84f3e807c6339559f7930: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260810 07:15:13.334218 23570 raft_consensus.cc:1261] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [term 1 FOLLOWER]: Refusing update from remote peer 29e10cc613d84f3e807c6339559f7930: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260810 07:15:13.334371 24297 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260810 07:15:13.334463 24325 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260810 07:15:13.335129 24298 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, c2d2d5a77d9f482f93cc231f8b7e5324 (127.15.217.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: false } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: false } } }
I20260810 07:15:13.335283 23570 raft_consensus.cc:2932] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, c2d2d5a77d9f482f93cc231f8b7e5324 (127.15.217.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: false } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: false } } }
I20260810 07:15:13.336411 24298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "29e10cc613d84f3e807c6339559f7930" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: false } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: false } } }
I20260810 07:15:13.336530 24298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:13.336589 24298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "29e10cc613d84f3e807c6339559f7930" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: false } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: false } } }
I20260810 07:15:13.336644 24298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:13.336629 24326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "29e10cc613d84f3e807c6339559f7930" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: false } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: false } } }
I20260810 07:15:13.336831 24326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:13.336810 24328 sys_catalog.cc:455] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "29e10cc613d84f3e807c6339559f7930" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: false } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: false } } }
I20260810 07:15:13.336913 24328 sys_catalog.cc:458] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:15:13.336856 23785 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, c2d2d5a77d9f482f93cc231f8b7e5324 (127.15.217.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: false } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: false } } }
I20260810 07:15:13.337888 24319 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "29e10cc613d84f3e807c6339559f7930" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: false } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: false } } }
I20260810 07:15:13.337990 24319 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:15:13.358743 24306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:13.356127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37900 (local address 127.15.217.66:46239)
0810 07:15:13.356220 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:13.356222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:13.356233 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:15:13.356359 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:13.356360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:13.356382 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:13.356480 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:15:13.356626 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.357114 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.357908 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.358111 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.358209 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:13.358388 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:15:13.358504 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:15:13.358604 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:13.358624 (+ 20us) server_negotiation.cc:299] Negotiation successful
0810 07:15:13.358649 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:15:13.361351 23920 raft_consensus.cc:1261] T a3d4d72c0568419b8b94becc36f0e3f0 P 2a7624fba8924223967850a43cc913c2 [term 1 FOLLOWER]: Refusing update from remote peer 87f6de6d8282454f9da57f1a17b89e6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:13.361421 24222 raft_consensus.cc:1261] T a3d4d72c0568419b8b94becc36f0e3f0 P ceed6041617a45b6ab60d5fb2064d12d [term 1 FOLLOWER]: Refusing update from remote peer 87f6de6d8282454f9da57f1a17b89e6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:13.361811 24317 consensus_queue.cc:1035] T a3d4d72c0568419b8b94becc36f0e3f0 P 87f6de6d8282454f9da57f1a17b89e6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceed6041617a45b6ab60d5fb2064d12d" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:13.362051 24323 consensus_queue.cc:1035] T a3d4d72c0568419b8b94becc36f0e3f0 P 87f6de6d8282454f9da57f1a17b89e6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a7624fba8924223967850a43cc913c2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:13.367779 24331 mvcc.cc:205] Tried to move back new op lower bound from 7316873680324829184 to 7316873680202092544. Current Snapshot: MvccSnapshot[applied={T|T < 7316873680324829184 or (T in {7316873680324829184})}]
I20260810 07:15:13.369258 22907 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51222:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:13.369413 22907 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51222:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:13.371526 22907 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 29e10cc613d84f3e807c6339559f7930: Sending DeleteTablet for 3 replicas of tablet a3d4d72c0568419b8b94becc36f0e3f0
I20260810 07:15:13.371953 24051 tablet_service.cc:1514] Processing DeleteTablet for tablet a3d4d72c0568419b8b94becc36f0e3f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:13 UTC) from {username='oryx', principal='oryx/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:37898
I20260810 07:15:13.372025 24199 tablet_service.cc:1514] Processing DeleteTablet for tablet a3d4d72c0568419b8b94becc36f0e3f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:13 UTC) from {username='oryx', principal='oryx/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:42584
I20260810 07:15:13.372488 23899 tablet_service.cc:1514] Processing DeleteTablet for tablet a3d4d72c0568419b8b94becc36f0e3f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:13 UTC) from {username='oryx', principal='oryx/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:34474
I20260810 07:15:13.372538 24342 tablet_replica.cc:330] T a3d4d72c0568419b8b94becc36f0e3f0 P ceed6041617a45b6ab60d5fb2064d12d: stopping tablet replica
I20260810 07:15:13.372660 24342 raft_consensus.cc:2229] T a3d4d72c0568419b8b94becc36f0e3f0 P ceed6041617a45b6ab60d5fb2064d12d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:13.372777 24342 raft_consensus.cc:2258] T a3d4d72c0568419b8b94becc36f0e3f0 P ceed6041617a45b6ab60d5fb2064d12d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:13.373176 24341 tablet_replica.cc:330] T a3d4d72c0568419b8b94becc36f0e3f0 P 87f6de6d8282454f9da57f1a17b89e6e: stopping tablet replica
I20260810 07:15:13.373291 24341 raft_consensus.cc:2229] T a3d4d72c0568419b8b94becc36f0e3f0 P 87f6de6d8282454f9da57f1a17b89e6e [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:15:13.373307 24345 tablet_replica.cc:330] T a3d4d72c0568419b8b94becc36f0e3f0 P 2a7624fba8924223967850a43cc913c2: stopping tablet replica
I20260810 07:15:13.373395 24341 raft_consensus.cc:2258] T a3d4d72c0568419b8b94becc36f0e3f0 P 87f6de6d8282454f9da57f1a17b89e6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:13.373386 24342 ts_tablet_manager.cc:1905] T a3d4d72c0568419b8b94becc36f0e3f0 P ceed6041617a45b6ab60d5fb2064d12d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:13.373449 24345 raft_consensus.cc:2229] T a3d4d72c0568419b8b94becc36f0e3f0 P 2a7624fba8924223967850a43cc913c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:13.373567 24345 raft_consensus.cc:2258] T a3d4d72c0568419b8b94becc36f0e3f0 P 2a7624fba8924223967850a43cc913c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:13.373620 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 22871
I20260810 07:15:13.378298 24341 ts_tablet_manager.cc:1905] T a3d4d72c0568419b8b94becc36f0e3f0 P 87f6de6d8282454f9da57f1a17b89e6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:13.379554 24342 ts_tablet_manager.cc:1918] T a3d4d72c0568419b8b94becc36f0e3f0 P ceed6041617a45b6ab60d5fb2064d12d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:15:13.379652 24342 log.cc:1196] T a3d4d72c0568419b8b94becc36f0e3f0 P ceed6041617a45b6ab60d5fb2064d12d: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-2/wal/wals/a3d4d72c0568419b8b94becc36f0e3f0
I20260810 07:15:13.379953 24342 ts_tablet_manager.cc:1939] T a3d4d72c0568419b8b94becc36f0e3f0 P ceed6041617a45b6ab60d5fb2064d12d: Deleting consensus metadata
I20260810 07:15:13.381125 24341 ts_tablet_manager.cc:1918] T a3d4d72c0568419b8b94becc36f0e3f0 P 87f6de6d8282454f9da57f1a17b89e6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:15:13.381294 24341 log.cc:1196] T a3d4d72c0568419b8b94becc36f0e3f0 P 87f6de6d8282454f9da57f1a17b89e6e: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-1/wal/wals/a3d4d72c0568419b8b94becc36f0e3f0
I20260810 07:15:13.381906 24341 ts_tablet_manager.cc:1939] T a3d4d72c0568419b8b94becc36f0e3f0 P 87f6de6d8282454f9da57f1a17b89e6e: Deleting consensus metadata
I20260810 07:15:13.385619 24294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:13.372676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34892 (local address 127.15.217.124:41403)
0810 07:15:13.372722 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:13.372724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:13.373189 (+ 465us) server_negotiation.cc:407] Connection header received
0810 07:15:13.373218 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:13.373219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:13.373231 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:13.373294 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:13.378005 (+ 4711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.378735 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.380069 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.380393 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.380445 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:13.382810 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:13.382811 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:13.382812 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:13.382820 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:13.384339 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:13.384766 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:13.384767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:13.384767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:13.384827 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:13.385063 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:13.385064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:13.385064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:13.385186 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:13.385296 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:13.385405 (+ 109us) server_negotiation.cc:299] Negotiation successful
0810 07:15:13.385505 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260810 07:15:13.386252 24150 connection.cc:535] server connection from 127.0.0.1:42584 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 07:15:13.386639 24000 connection.cc:535] server connection from 127.0.0.1:37898 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 07:15:13.387642 24345 ts_tablet_manager.cc:1905] T a3d4d72c0568419b8b94becc36f0e3f0 P 2a7624fba8924223967850a43cc913c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:13.389807 24345 ts_tablet_manager.cc:1918] T a3d4d72c0568419b8b94becc36f0e3f0 P 2a7624fba8924223967850a43cc913c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:15:13.389896 24345 log.cc:1196] T a3d4d72c0568419b8b94becc36f0e3f0 P 2a7624fba8924223967850a43cc913c2: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-0/wal/wals/a3d4d72c0568419b8b94becc36f0e3f0
I20260810 07:15:13.390216 24345 ts_tablet_manager.cc:1939] T a3d4d72c0568419b8b94becc36f0e3f0 P 2a7624fba8924223967850a43cc913c2: Deleting consensus metadata
W20260810 07:15:13.390647 23851 connection.cc:441] server connection from 127.0.0.1:34474 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:34474 (request call id 1) could send its response
I20260810 07:15:13.396059 24294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:13.388395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34894 (local address 127.15.217.124:41403)
0810 07:15:13.388433 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:13.388435 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:13.388442 (+ 7us) server_negotiation.cc:407] Connection header received
0810 07:15:13.388482 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:13.388483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:13.388493 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:13.388552 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:13.389269 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.389909 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.391316 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.391536 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.391618 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:13.393325 (+ 1707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:13.393326 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:13.393326 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:13.393333 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:13.394895 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:13.395243 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:13.395244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:13.395244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:13.395307 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:13.395521 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:13.395522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:13.395522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:13.395642 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:13.395744 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:13.395853 (+ 109us) server_negotiation.cc:299] Negotiation successful
0810 07:15:13.395932 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 07:15:13.403295 24292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:13.388682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48316 (local address 127.15.217.125:44791)
0810 07:15:13.388740 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:13.388742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:13.396072 (+ 7330us) server_negotiation.cc:407] Connection header received
0810 07:15:13.396470 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:13.396472 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:13.396493 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:13.396555 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:13.397233 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.397717 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.398726 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.398920 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.398971 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:13.400409 (+ 1438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:13.400411 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:13.400411 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:13.400418 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:13.401983 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:13.402468 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:13.402469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:13.402469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:13.402529 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:13.402830 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:13.402830 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:13.402830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:13.402939 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:13.403013 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:13.403152 (+ 139us) server_negotiation.cc:299] Negotiation successful
0810 07:15:13.403225 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 07:15:14.367435 24353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.367126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41187 (local address 127.15.217.66:48365)
0810 07:15:14.367267 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:14.367367 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:41187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":65,"threads_started":1}
I20260810 07:15:14.367595 24352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.367089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41187 (local address 127.15.217.67:36847)
0810 07:15:14.367408 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:14.367502 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:41187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":83,"threads_started":1}
I20260810 07:15:14.367643 24354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.367395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41187 (local address 127.15.217.65:45213)
0810 07:15:14.367504 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:14.367596 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:41187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":54,"threads_started":1}
W20260810 07:15:14.367677 24127 heartbeater.cc:644] Failed to heartbeat to 127.15.217.126:41187 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.217.126:41187: connect: Connection refused (error 111)
W20260810 07:15:14.367820 23974 heartbeater.cc:644] Failed to heartbeat to 127.15.217.126:41187 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.217.126:41187: connect: Connection refused (error 111)
W20260810 07:15:14.368361 24279 heartbeater.cc:644] Failed to heartbeat to 127.15.217.126:41187 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.217.126:41187: connect: Connection refused (error 111)
I20260810 07:15:14.871699 24355 raft_consensus.cc:482] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 29e10cc613d84f3e807c6339559f7930)
I20260810 07:15:14.871856 24355 raft_consensus.cc:504] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: false } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: false } }
I20260810 07:15:14.872109 24355 leader_election.cc:288] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29e10cc613d84f3e807c6339559f7930 (127.15.217.126:41187), c2d2d5a77d9f482f93cc231f8b7e5324 (127.15.217.124:41403)
I20260810 07:15:14.872418 24356 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 29e10cc613d84f3e807c6339559f7930)
I20260810 07:15:14.872597 24357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.872291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41187 (local address 127.0.0.1:51684)
0810 07:15:14.872428 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:14.872542 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:41187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":79,"threads_started":1}
W20260810 07:15:14.872764 23544 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.217.126:41187: connect: Connection refused (error 111)
I20260810 07:15:14.872779 24356 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: false } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: false } }
I20260810 07:15:14.873222 24357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.873114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41187 (local address 127.0.0.1:51686)
0810 07:15:14.873156 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:14.873188 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:41187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260810 07:15:14.873353 23544 leader_election.cc:334] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 29e10cc613d84f3e807c6339559f7930 (127.15.217.126:41187): Network error: Client connection negotiation failed: client connection to 127.15.217.126:41187: connect: Connection refused (error 111)
I20260810 07:15:14.873443 24356 leader_election.cc:288] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 29e10cc613d84f3e807c6339559f7930 (127.15.217.126:41187), be6a5e4564f448cda7ea84840d63d409 (127.15.217.125:44791)
I20260810 07:15:14.873587 24361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.873162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41187 (local address 127.0.0.1:51694)
0810 07:15:14.873369 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:14.873485 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:41187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":47,"threads_started":1}
W20260810 07:15:14.873725 23758 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.217.126:41187: connect: Connection refused (error 111)
I20260810 07:15:14.874150 24361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.874059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41187 (local address 127.0.0.1:51696)
0810 07:15:14.874093 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:14.874123 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:41187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260810 07:15:14.874274 23758 leader_election.cc:334] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 29e10cc613d84f3e807c6339559f7930 (127.15.217.126:41187): Network error: Client connection negotiation failed: client connection to 127.15.217.126:41187: connect: Connection refused (error 111)
I20260810 07:15:14.876765 24358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.872538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:41403 (local address 127.0.0.1:34898)
0810 07:15:14.872655 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:14.872952 (+ 297us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:14.873032 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:14.873191 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:14.873193 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:14.873195 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:14.873262 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:14.873265 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.874392 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.874393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:14.875692 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:14.875696 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.876429 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.876430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:14.876535 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:14.876540 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:15:14.876618 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 07:15:14.876672 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
I20260810 07:15:14.876773 24359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.872653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34898 (local address 127.15.217.124:41403)
0810 07:15:14.872753 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:14.872756 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:14.873043 (+ 287us) server_negotiation.cc:407] Connection header received
0810 07:15:14.873090 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:14.873092 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:14.873104 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:14.873177 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:14.873524 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.874301 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.875786 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.876344 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.876423 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:14.876433 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:14.876634 (+ 201us) server_negotiation.cc:299] Negotiation successful
0810 07:15:14.876690 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78,"thread_start_us":46,"threads_started":1}
I20260810 07:15:14.876895 24362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.873370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:44791 (local address 127.0.0.1:48320)
0810 07:15:14.873551 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:14.873562 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:14.873620 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:14.873820 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:14.873821 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:14.873823 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:14.873876 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:14.873877 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.874710 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.874712 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:14.875857 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:14.875860 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.876642 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.876643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:14.876702 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:14.876706 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:15:14.876761 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 07:15:14.876807 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":48,"threads_started":1}
I20260810 07:15:14.876987 23785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "be6a5e4564f448cda7ea84840d63d409" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" is_pre_election: true
I20260810 07:15:14.877089 24363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.873408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48320 (local address 127.15.217.125:44791)
0810 07:15:14.873555 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:14.873557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:14.873604 (+ 47us) server_negotiation.cc:407] Connection header received
0810 07:15:14.873696 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:14.873697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:14.873707 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:14.873812 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:14.873966 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.874631 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.875933 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.876549 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.876650 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:14.876658 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:14.876776 (+ 118us) server_negotiation.cc:299] Negotiation successful
0810 07:15:14.877028 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":88,"threads_started":1}
I20260810 07:15:14.877094 23785 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be6a5e4564f448cda7ea84840d63d409 in term 1.
I20260810 07:15:14.877234 23544 leader_election.cc:302] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: be6a5e4564f448cda7ea84840d63d409, c2d2d5a77d9f482f93cc231f8b7e5324; no voters: 29e10cc613d84f3e807c6339559f7930
I20260810 07:15:14.877275 23570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "be6a5e4564f448cda7ea84840d63d409" is_pre_election: true
I20260810 07:15:14.877347 24355 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 07:15:14.877434 24355 raft_consensus.cc:482] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 29e10cc613d84f3e807c6339559f7930)
I20260810 07:15:14.877452 24355 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:15:14.878216 24355 raft_consensus.cc:504] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: false } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: false } }
I20260810 07:15:14.878351 23570 raft_consensus.cc:2379] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c2d2d5a77d9f482f93cc231f8b7e5324 in current term 2: Already voted for candidate be6a5e4564f448cda7ea84840d63d409 in this term.
I20260810 07:15:14.878360 24355 leader_election.cc:288] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [CANDIDATE]: Term 2 election: Requested vote from peers 29e10cc613d84f3e807c6339559f7930 (127.15.217.126:41187), c2d2d5a77d9f482f93cc231f8b7e5324 (127.15.217.124:41403)
I20260810 07:15:14.878531 24358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.878446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41187 (local address 127.0.0.1:51702)
0810 07:15:14.878478 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:14.878505 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:41187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260810 07:15:14.878607 23785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "be6a5e4564f448cda7ea84840d63d409" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324"
I20260810 07:15:14.878693 23785 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:15:14.878785 23760 leader_election.cc:302] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c2d2d5a77d9f482f93cc231f8b7e5324; no voters: 29e10cc613d84f3e807c6339559f7930, be6a5e4564f448cda7ea84840d63d409
I20260810 07:15:14.879405 23785 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be6a5e4564f448cda7ea84840d63d409 in term 2.
I20260810 07:15:14.879487 24356 raft_consensus.cc:2726] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:15:14.879582 23544 leader_election.cc:302] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be6a5e4564f448cda7ea84840d63d409, c2d2d5a77d9f482f93cc231f8b7e5324; no voters:
I20260810 07:15:14.879703 24355 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 07:15:14.879848 24355 raft_consensus.cc:686] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [term 2 LEADER]: Becoming Leader. State: Replica: be6a5e4564f448cda7ea84840d63d409, State: Running, Role: LEADER
I20260810 07:15:14.879887 24358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.879767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41187 (local address 127.0.0.1:51712)
0810 07:15:14.879833 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:14.879866 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:41187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 07:15:14.879974 23544 leader_election.cc:334] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 29e10cc613d84f3e807c6339559f7930 (127.15.217.126:41187): Network error: Client connection negotiation failed: client connection to 127.15.217.126:41187: connect: Connection refused (error 111)
I20260810 07:15:14.879936 24355 consensus_queue.cc:237] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: false } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: false } }
I20260810 07:15:14.880302 24365 sys_catalog.cc:455] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be6a5e4564f448cda7ea84840d63d409. Latest consensus state: current_term: 2 leader_uuid: "be6a5e4564f448cda7ea84840d63d409" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: false } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: false } } }
I20260810 07:15:14.880405 24365 sys_catalog.cc:458] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:14.880620 24367 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:14.881028 24367 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:15:14.881088 24367 catalog_manager.cc:1259] Loaded cluster ID: 213dbfc3d5314eb3b1002679033b6e06
I20260810 07:15:14.881137 24367 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:15:14.881314 24367 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:14.881417 24367 catalog_manager.cc:5879] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409: Loaded TSK: 0
I20260810 07:15:14.881567 24367 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:15:14.892674 23559 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48316:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:15:14.894641 23559 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:15:14.895685 24358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.895580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41187 (local address 127.0.0.1:51724)
0810 07:15:14.895628 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:14.895659 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:41187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260810 07:15:14.895821 23785 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [term 2 FOLLOWER]: Refusing update from remote peer be6a5e4564f448cda7ea84840d63d409: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260810 07:15:14.896066 24358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.895966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41187 (local address 127.0.0.1:51726)
0810 07:15:14.896005 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:14.896034 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:41187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260810 07:15:14.896142 23544 consensus_peers.cc:487] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 -> Peer 29e10cc613d84f3e807c6339559f7930 (127.15.217.126:41187): Couldn't send request to peer 29e10cc613d84f3e807c6339559f7930. Status: Network error: Client connection negotiation failed: client connection to 127.15.217.126:41187: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 07:15:14.896173 24355 consensus_queue.cc:1035] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260810 07:15:14.896860 24356 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be6a5e4564f448cda7ea84840d63d409. Latest consensus state: current_term: 2 leader_uuid: "be6a5e4564f448cda7ea84840d63d409" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: false } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: false } } }
I20260810 07:15:14.896973 24356 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:15:14.897401 24368 mvcc.cc:205] Tried to move back new op lower bound from 7316873686610989056 to 7316873686548946944. Current Snapshot: MvccSnapshot[applied={T|T < 7316873686610989056}]
I20260810 07:15:14.897533 24365 sys_catalog.cc:455] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "be6a5e4564f448cda7ea84840d63d409" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: false } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: false } } }
I20260810 07:15:14.897637 24365 sys_catalog.cc:458] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:14.897779 24356 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "be6a5e4564f448cda7ea84840d63d409" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "29e10cc613d84f3e807c6339559f7930" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41187 } } peers { permanent_uuid: "be6a5e4564f448cda7ea84840d63d409" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 44791 } attrs { promote: false } } peers { permanent_uuid: "c2d2d5a77d9f482f93cc231f8b7e5324" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 41403 } attrs { promote: false } } }
I20260810 07:15:14.897867 24356 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2d2d5a77d9f482f93cc231f8b7e5324 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:15:14.902827 24358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.899446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:46239 (local address 127.0.0.1:37908)
0810 07:15:14.899662 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:14.899672 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:14.899706 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:14.900029 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:14.900030 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:14.900031 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:14.900057 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:14.900058 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.901035 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.901036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:14.901826 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:14.901829 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.902564 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.902565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:14.902627 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:14.902634 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 07:15:14.902695 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 07:15:14.902743 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203}
I20260810 07:15:14.902849 24378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.899538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37908 (local address 127.15.217.66:46239)
0810 07:15:14.899892 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:14.899894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:14.899902 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:15:14.899929 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:14.899930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:14.899940 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:14.900120 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:14.900173 (+ 53us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.900921 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.901927 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.902473 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.902563 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:14.902577 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:14.902718 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 07:15:14.902764 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":136,"threads_started":1}
I20260810 07:15:14.903044 24375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.899521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:34201 (local address 127.0.0.1:34478)
0810 07:15:14.899721 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:14.899733 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:14.899792 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:14.900085 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:14.900085 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:14.900088 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:14.900138 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:14.900139 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.900872 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.900873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:14.902119 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:14.902122 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.902811 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.902812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:14.902871 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:14.902876 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:15:14.902924 (+ 48us) client_negotiation.cc:240] Negotiation successful
0810 07:15:14.902979 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":2,"thread_start_us":106,"threads_started":1}
I20260810 07:15:14.903160 24357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.899427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:39201 (local address 127.0.0.1:42588)
0810 07:15:14.899505 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:14.899516 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:14.899567 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:14.899909 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:14.899910 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:14.899914 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:14.899982 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:14.899984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.901004 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.901004 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:14.902123 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:14.902125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.902913 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.902915 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:14.903003 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:14.903007 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:15:14.903035 (+ 28us) client_negotiation.cc:240] Negotiation successful
0810 07:15:14.903085 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":44}
I20260810 07:15:14.903258 24051 tablet_service.cc:1467] Processing CreateTablet for tablet 2bf4f108cfee4559a9f82f95024ff33d (DEFAULT_TABLE table=test-table [id=d85a7239865e4da786e99f8e3ef41b3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:14.903379 24379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.899732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34478 (local address 127.15.217.65:34201)
0810 07:15:14.899848 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:14.899850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:14.899858 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:15:14.899887 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:14.899888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:14.899900 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:14.900026 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:14.900217 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.900789 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.902226 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.902709 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.902768 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:14.902779 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:14.903242 (+ 463us) server_negotiation.cc:299] Negotiation successful
0810 07:15:14.903310 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":65,"threads_started":1}
I20260810 07:15:14.903420 24051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bf4f108cfee4559a9f82f95024ff33d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:14.903501 24373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.899530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42588 (local address 127.15.217.67:39201)
0810 07:15:14.899756 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:14.899758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:14.899767 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:15:14.899798 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:14.899799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:14.899811 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:14.900050 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:14.900093 (+ 43us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.900926 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.902222 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.902810 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.903345 (+ 535us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:14.903363 (+ 18us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:14.903387 (+ 24us) server_negotiation.cc:299] Negotiation successful
0810 07:15:14.903435 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":54,"threads_started":1}
I20260810 07:15:14.904589 24199 tablet_service.cc:1467] Processing CreateTablet for tablet 2bf4f108cfee4559a9f82f95024ff33d (DEFAULT_TABLE table=test-table [id=d85a7239865e4da786e99f8e3ef41b3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:14.904739 24199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bf4f108cfee4559a9f82f95024ff33d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:14.905138 24381 tablet_bootstrap.cc:492] T 2bf4f108cfee4559a9f82f95024ff33d P 87f6de6d8282454f9da57f1a17b89e6e: Bootstrap starting.
I20260810 07:15:14.905571 24381 tablet_bootstrap.cc:654] T 2bf4f108cfee4559a9f82f95024ff33d P 87f6de6d8282454f9da57f1a17b89e6e: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:14.906051 24383 tablet_bootstrap.cc:492] T 2bf4f108cfee4559a9f82f95024ff33d P ceed6041617a45b6ab60d5fb2064d12d: Bootstrap starting.
I20260810 07:15:14.906097 23899 tablet_service.cc:1467] Processing CreateTablet for tablet 2bf4f108cfee4559a9f82f95024ff33d (DEFAULT_TABLE table=test-table [id=d85a7239865e4da786e99f8e3ef41b3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:14.906175 24381 tablet_bootstrap.cc:492] T 2bf4f108cfee4559a9f82f95024ff33d P 87f6de6d8282454f9da57f1a17b89e6e: No bootstrap required, opened a new log
I20260810 07:15:14.906226 23899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bf4f108cfee4559a9f82f95024ff33d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:14.906243 24381 ts_tablet_manager.cc:1397] T 2bf4f108cfee4559a9f82f95024ff33d P 87f6de6d8282454f9da57f1a17b89e6e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:14.906432 24383 tablet_bootstrap.cc:654] T 2bf4f108cfee4559a9f82f95024ff33d P ceed6041617a45b6ab60d5fb2064d12d: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:14.906391 24381 raft_consensus.cc:348] T 2bf4f108cfee4559a9f82f95024ff33d P 87f6de6d8282454f9da57f1a17b89e6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceed6041617a45b6ab60d5fb2064d12d" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39201 } } peers { permanent_uuid: "2a7624fba8924223967850a43cc913c2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34201 } } peers { permanent_uuid: "87f6de6d8282454f9da57f1a17b89e6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46239 } }
I20260810 07:15:14.906474 24381 raft_consensus.cc:374] T 2bf4f108cfee4559a9f82f95024ff33d P 87f6de6d8282454f9da57f1a17b89e6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:14.906498 24381 raft_consensus.cc:729] T 2bf4f108cfee4559a9f82f95024ff33d P 87f6de6d8282454f9da57f1a17b89e6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87f6de6d8282454f9da57f1a17b89e6e, State: Initialized, Role: FOLLOWER
I20260810 07:15:14.906584 24381 consensus_queue.cc:260] T 2bf4f108cfee4559a9f82f95024ff33d P 87f6de6d8282454f9da57f1a17b89e6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceed6041617a45b6ab60d5fb2064d12d" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39201 } } peers { permanent_uuid: "2a7624fba8924223967850a43cc913c2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34201 } } peers { permanent_uuid: "87f6de6d8282454f9da57f1a17b89e6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46239 } }
I20260810 07:15:14.906848 24381 ts_tablet_manager.cc:1428] T 2bf4f108cfee4559a9f82f95024ff33d P 87f6de6d8282454f9da57f1a17b89e6e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:14.907267 24387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.907023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41187 (local address 127.15.217.66:37739)
0810 07:15:14.907184 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:14.907230 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:41187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":91,"threads_started":1}
I20260810 07:15:14.907284 24383 tablet_bootstrap.cc:492] T 2bf4f108cfee4559a9f82f95024ff33d P ceed6041617a45b6ab60d5fb2064d12d: No bootstrap required, opened a new log
I20260810 07:15:14.907332 24383 ts_tablet_manager.cc:1397] T 2bf4f108cfee4559a9f82f95024ff33d P ceed6041617a45b6ab60d5fb2064d12d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:14.907459 24383 raft_consensus.cc:348] T 2bf4f108cfee4559a9f82f95024ff33d P ceed6041617a45b6ab60d5fb2064d12d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceed6041617a45b6ab60d5fb2064d12d" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39201 } } peers { permanent_uuid: "2a7624fba8924223967850a43cc913c2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34201 } } peers { permanent_uuid: "87f6de6d8282454f9da57f1a17b89e6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46239 } }
I20260810 07:15:14.907542 24383 raft_consensus.cc:374] T 2bf4f108cfee4559a9f82f95024ff33d P ceed6041617a45b6ab60d5fb2064d12d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:14.907648 24383 raft_consensus.cc:729] T 2bf4f108cfee4559a9f82f95024ff33d P ceed6041617a45b6ab60d5fb2064d12d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceed6041617a45b6ab60d5fb2064d12d, State: Initialized, Role: FOLLOWER
I20260810 07:15:14.907761 24383 consensus_queue.cc:260] T 2bf4f108cfee4559a9f82f95024ff33d P ceed6041617a45b6ab60d5fb2064d12d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceed6041617a45b6ab60d5fb2064d12d" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39201 } } peers { permanent_uuid: "2a7624fba8924223967850a43cc913c2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34201 } } peers { permanent_uuid: "87f6de6d8282454f9da57f1a17b89e6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46239 } }
I20260810 07:15:14.907936 24383 ts_tablet_manager.cc:1428] T 2bf4f108cfee4559a9f82f95024ff33d P ceed6041617a45b6ab60d5fb2064d12d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:14.908202 24387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.907718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41187 (local address 127.15.217.66:57443)
0810 07:15:14.908133 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:14.908161 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:41187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397}
I20260810 07:15:14.908524 24390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.908154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41187 (local address 127.15.217.67:51525)
0810 07:15:14.908405 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:14.908481 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:41187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":85,"threads_started":1}
I20260810 07:15:14.908667 24388 tablet_bootstrap.cc:492] T 2bf4f108cfee4559a9f82f95024ff33d P 2a7624fba8924223967850a43cc913c2: Bootstrap starting.
I20260810 07:15:14.908855 24390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.908772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41187 (local address 127.15.217.67:49625)
0810 07:15:14.908802 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:14.908832 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:41187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
I20260810 07:15:14.909041 24280 heartbeater.cc:503] Master 127.15.217.125:44791 was elected leader, sending a full tablet report...
I20260810 07:15:14.909054 24388 tablet_bootstrap.cc:654] T 2bf4f108cfee4559a9f82f95024ff33d P 2a7624fba8924223967850a43cc913c2: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:14.909667 24388 tablet_bootstrap.cc:492] T 2bf4f108cfee4559a9f82f95024ff33d P 2a7624fba8924223967850a43cc913c2: No bootstrap required, opened a new log
I20260810 07:15:14.909713 24388 ts_tablet_manager.cc:1397] T 2bf4f108cfee4559a9f82f95024ff33d P 2a7624fba8924223967850a43cc913c2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:14.909864 24388 raft_consensus.cc:348] T 2bf4f108cfee4559a9f82f95024ff33d P 2a7624fba8924223967850a43cc913c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceed6041617a45b6ab60d5fb2064d12d" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39201 } } peers { permanent_uuid: "2a7624fba8924223967850a43cc913c2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34201 } } peers { permanent_uuid: "87f6de6d8282454f9da57f1a17b89e6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46239 } }
I20260810 07:15:14.909934 24388 raft_consensus.cc:374] T 2bf4f108cfee4559a9f82f95024ff33d P 2a7624fba8924223967850a43cc913c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:14.909955 24388 raft_consensus.cc:729] T 2bf4f108cfee4559a9f82f95024ff33d P 2a7624fba8924223967850a43cc913c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a7624fba8924223967850a43cc913c2, State: Initialized, Role: FOLLOWER
I20260810 07:15:14.910024 24388 consensus_queue.cc:260] T 2bf4f108cfee4559a9f82f95024ff33d P 2a7624fba8924223967850a43cc913c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceed6041617a45b6ab60d5fb2064d12d" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39201 } } peers { permanent_uuid: "2a7624fba8924223967850a43cc913c2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34201 } } peers { permanent_uuid: "87f6de6d8282454f9da57f1a17b89e6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46239 } }
I20260810 07:15:14.910192 24388 ts_tablet_manager.cc:1428] T 2bf4f108cfee4559a9f82f95024ff33d P 2a7624fba8924223967850a43cc913c2: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 07:15:14.910596 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.910356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41187 (local address 127.15.217.65:35357)
0810 07:15:14.910518 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:14.910557 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:41187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":99,"threads_started":1}
I20260810 07:15:14.910678 23975 heartbeater.cc:503] Master 127.15.217.125:44791 was elected leader, sending a full tablet report...
I20260810 07:15:14.911022 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.910912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41187 (local address 127.15.217.65:33967)
0810 07:15:14.910964 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:14.910992 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:41187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260810 07:15:14.964403 24392 raft_consensus.cc:482] T 2bf4f108cfee4559a9f82f95024ff33d P 2a7624fba8924223967850a43cc913c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:15:14.964564 24392 raft_consensus.cc:504] T 2bf4f108cfee4559a9f82f95024ff33d P 2a7624fba8924223967850a43cc913c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceed6041617a45b6ab60d5fb2064d12d" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39201 } } peers { permanent_uuid: "2a7624fba8924223967850a43cc913c2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34201 } } peers { permanent_uuid: "87f6de6d8282454f9da57f1a17b89e6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46239 } }
I20260810 07:15:14.964906 24392 leader_election.cc:288] T 2bf4f108cfee4559a9f82f95024ff33d P 2a7624fba8924223967850a43cc913c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceed6041617a45b6ab60d5fb2064d12d (127.15.217.67:39201), 87f6de6d8282454f9da57f1a17b89e6e (127.15.217.66:46239)
I20260810 07:15:14.968308 24393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.965141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:46239 (local address 127.15.217.65:53787)
0810 07:15:14.965213 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:14.965226 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:14.965284 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:14.965485 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:14.965487 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:14.965490 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:14.965595 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:14.965598 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.966490 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.966491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:14.967350 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:14.967352 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.968039 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.968040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:14.968121 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:14.968126 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:15:14.968175 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 07:15:14.968207 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 07:15:14.968330 24378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.965227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:53787 (local address 127.15.217.66:46239)
0810 07:15:14.965281 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:14.965283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:14.965297 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:15:14.965368 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:14.965369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:14.965380 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:14.965443 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:14.965719 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.966398 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.967446 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.967954 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.968054 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:14.968068 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:14.968209 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 07:15:14.968259 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 07:15:14.968395 24373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.965228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:47589 (local address 127.15.217.67:39201)
0810 07:15:14.965293 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:14.965295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:14.965494 (+ 199us) server_negotiation.cc:407] Connection header received
0810 07:15:14.965533 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:14.965534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:14.965547 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:14.965604 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:14.965881 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.966511 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.967582 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.967958 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.968049 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:14.968060 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:14.968281 (+ 221us) server_negotiation.cc:299] Negotiation successful
0810 07:15:14.968325 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 07:15:14.968662 24394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.965143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:39201 (local address 127.15.217.65:47589)
0810 07:15:14.965376 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:14.965388 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:14.965450 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:14.965719 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:14.965720 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:14.965722 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:14.965791 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:14.965792 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.966633 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.966634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:14.967470 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:14.967472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.968076 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.968077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:14.968193 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:14.968198 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:15:14.968257 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 07:15:14.968296 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":24,"thread_start_us":126,"threads_started":1}
I20260810 07:15:14.968696 24072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bf4f108cfee4559a9f82f95024ff33d" candidate_uuid: "2a7624fba8924223967850a43cc913c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87f6de6d8282454f9da57f1a17b89e6e" is_pre_election: true
I20260810 07:15:14.968911 24072 raft_consensus.cc:2454] T 2bf4f108cfee4559a9f82f95024ff33d P 87f6de6d8282454f9da57f1a17b89e6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a7624fba8924223967850a43cc913c2 in term 0.
I20260810 07:15:14.969053 24222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bf4f108cfee4559a9f82f95024ff33d" candidate_uuid: "2a7624fba8924223967850a43cc913c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceed6041617a45b6ab60d5fb2064d12d" is_pre_election: true
I20260810 07:15:14.969161 24222 raft_consensus.cc:2454] T 2bf4f108cfee4559a9f82f95024ff33d P ceed6041617a45b6ab60d5fb2064d12d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a7624fba8924223967850a43cc913c2 in term 0.
I20260810 07:15:14.969139 23850 leader_election.cc:302] T 2bf4f108cfee4559a9f82f95024ff33d P 2a7624fba8924223967850a43cc913c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a7624fba8924223967850a43cc913c2, 87f6de6d8282454f9da57f1a17b89e6e; no voters:
I20260810 07:15:14.969327 24392 raft_consensus.cc:2781] T 2bf4f108cfee4559a9f82f95024ff33d P 2a7624fba8924223967850a43cc913c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:15:14.969377 24392 raft_consensus.cc:482] T 2bf4f108cfee4559a9f82f95024ff33d P 2a7624fba8924223967850a43cc913c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:15:14.969398 24392 raft_consensus.cc:3037] T 2bf4f108cfee4559a9f82f95024ff33d P 2a7624fba8924223967850a43cc913c2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:14.970180 24392 raft_consensus.cc:504] T 2bf4f108cfee4559a9f82f95024ff33d P 2a7624fba8924223967850a43cc913c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceed6041617a45b6ab60d5fb2064d12d" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39201 } } peers { permanent_uuid: "2a7624fba8924223967850a43cc913c2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34201 } } peers { permanent_uuid: "87f6de6d8282454f9da57f1a17b89e6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46239 } }
I20260810 07:15:14.970332 24392 leader_election.cc:288] T 2bf4f108cfee4559a9f82f95024ff33d P 2a7624fba8924223967850a43cc913c2 [CANDIDATE]: Term 1 election: Requested vote from peers ceed6041617a45b6ab60d5fb2064d12d (127.15.217.67:39201), 87f6de6d8282454f9da57f1a17b89e6e (127.15.217.66:46239)
I20260810 07:15:14.970541 24222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bf4f108cfee4559a9f82f95024ff33d" candidate_uuid: "2a7624fba8924223967850a43cc913c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceed6041617a45b6ab60d5fb2064d12d"
I20260810 07:15:14.970534 24072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bf4f108cfee4559a9f82f95024ff33d" candidate_uuid: "2a7624fba8924223967850a43cc913c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87f6de6d8282454f9da57f1a17b89e6e"
I20260810 07:15:14.970619 24072 raft_consensus.cc:3037] T 2bf4f108cfee4559a9f82f95024ff33d P 87f6de6d8282454f9da57f1a17b89e6e [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:14.970621 24222 raft_consensus.cc:3037] T 2bf4f108cfee4559a9f82f95024ff33d P ceed6041617a45b6ab60d5fb2064d12d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:14.971202 24222 raft_consensus.cc:2454] T 2bf4f108cfee4559a9f82f95024ff33d P ceed6041617a45b6ab60d5fb2064d12d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a7624fba8924223967850a43cc913c2 in term 1.
I20260810 07:15:14.971403 23851 leader_election.cc:302] T 2bf4f108cfee4559a9f82f95024ff33d P 2a7624fba8924223967850a43cc913c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a7624fba8924223967850a43cc913c2, ceed6041617a45b6ab60d5fb2064d12d; no voters:
I20260810 07:15:14.971505 24072 raft_consensus.cc:2454] T 2bf4f108cfee4559a9f82f95024ff33d P 87f6de6d8282454f9da57f1a17b89e6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a7624fba8924223967850a43cc913c2 in term 1.
I20260810 07:15:14.971541 24392 raft_consensus.cc:2781] T 2bf4f108cfee4559a9f82f95024ff33d P 2a7624fba8924223967850a43cc913c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:14.971740 24392 raft_consensus.cc:686] T 2bf4f108cfee4559a9f82f95024ff33d P 2a7624fba8924223967850a43cc913c2 [term 1 LEADER]: Becoming Leader. State: Replica: 2a7624fba8924223967850a43cc913c2, State: Running, Role: LEADER
I20260810 07:15:14.971843 24392 consensus_queue.cc:237] T 2bf4f108cfee4559a9f82f95024ff33d P 2a7624fba8924223967850a43cc913c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceed6041617a45b6ab60d5fb2064d12d" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39201 } } peers { permanent_uuid: "2a7624fba8924223967850a43cc913c2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34201 } } peers { permanent_uuid: "87f6de6d8282454f9da57f1a17b89e6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46239 } }
I20260810 07:15:14.972085 24394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.971988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41187 (local address 127.15.217.65:33101)
0810 07:15:14.972023 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:14.972059 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:41187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260810 07:15:14.972652 23559 catalog_manager.cc:5495] T 2bf4f108cfee4559a9f82f95024ff33d P 2a7624fba8924223967850a43cc913c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a7624fba8924223967850a43cc913c2 (127.15.217.65). New cstate: current_term: 1 leader_uuid: "2a7624fba8924223967850a43cc913c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ceed6041617a45b6ab60d5fb2064d12d" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39201 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a7624fba8924223967850a43cc913c2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34201 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "87f6de6d8282454f9da57f1a17b89e6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46239 } health_report { overall_health: UNKNOWN } } }
I20260810 07:15:14.996109 24379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.993558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34488 (local address 127.15.217.65:34201)
0810 07:15:14.993610 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:14.993612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:14.993621 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:15:14.993708 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:14.993709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:14.993730 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:14.993828 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:15:14.993954 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.994644 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.995432 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.995625 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.995685 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:14.995799 (+ 114us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:15:14.995909 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:15:14.995960 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:14.995999 (+ 39us) server_negotiation.cc:299] Negotiation successful
0810 07:15:14.996018 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 07:15:14.997496 24222 raft_consensus.cc:1261] T 2bf4f108cfee4559a9f82f95024ff33d P ceed6041617a45b6ab60d5fb2064d12d [term 1 FOLLOWER]: Refusing update from remote peer 2a7624fba8924223967850a43cc913c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:14.997584 24072 raft_consensus.cc:1261] T 2bf4f108cfee4559a9f82f95024ff33d P 87f6de6d8282454f9da57f1a17b89e6e [term 1 FOLLOWER]: Refusing update from remote peer 2a7624fba8924223967850a43cc913c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:14.997864 24392 consensus_queue.cc:1035] T 2bf4f108cfee4559a9f82f95024ff33d P 2a7624fba8924223967850a43cc913c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceed6041617a45b6ab60d5fb2064d12d" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:14.998047 24395 consensus_queue.cc:1035] T 2bf4f108cfee4559a9f82f95024ff33d P 2a7624fba8924223967850a43cc913c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87f6de6d8282454f9da57f1a17b89e6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:14.998622 24128 heartbeater.cc:503] Master 127.15.217.125:44791 was elected leader, sending a full tablet report...
I20260810 07:15:14.999019 24387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.998898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41187 (local address 127.15.217.66:35515)
0810 07:15:14.998949 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:14.998984 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:41187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260810 07:15:14.999871 24390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.999715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41187 (local address 127.15.217.67:36905)
0810 07:15:14.999794 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:14.999835 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:41187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260810 07:15:15.000620 24402 mvcc.cc:205] Tried to move back new op lower bound from 7316873687027036160 to 7316873686925365248. Current Snapshot: MvccSnapshot[applied={T|T < 7316873687027036160}]
I20260810 07:15:15.000725 24394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:15.000511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41187 (local address 127.15.217.65:39469)
0810 07:15:15.000660 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:15.000693 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:41187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260810 07:15:15.000749 24387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:15.000656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41187 (local address 127.15.217.66:58735)
0810 07:15:15.000692 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:15.000726 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:41187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260810 07:15:15.001487 24394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:15.001374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41187 (local address 127.15.217.65:53085)
0810 07:15:15.001439 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:15.001462 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:41187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 07:15:15.002305 24390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:15.002179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41187 (local address 127.15.217.67:44693)
0810 07:15:15.002238 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:15.002274 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:41187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 07:15:15.002028 24400 mvcc.cc:205] Tried to move back new op lower bound from 7316873687027036160 to 7316873686925365248. Current Snapshot: MvccSnapshot[applied={T|T < 7316873687027036160}]
I20260810 07:15:15.002820 23559 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48316:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:15.002925 23559 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48316:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:15.004442 23559 catalog_manager.cc:5782] T 00000000000000000000000000000000 P be6a5e4564f448cda7ea84840d63d409: Sending DeleteTablet for 3 replicas of tablet 2bf4f108cfee4559a9f82f95024ff33d
I20260810 07:15:15.004779 24199 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bf4f108cfee4559a9f82f95024ff33d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:15 UTC) from {username='oryx', principal='oryx/127.15.217.125@KRBTEST.COM'} at 127.0.0.1:42588
I20260810 07:15:15.004817 23899 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bf4f108cfee4559a9f82f95024ff33d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:15 UTC) from {username='oryx', principal='oryx/127.15.217.125@KRBTEST.COM'} at 127.0.0.1:34478
I20260810 07:15:15.005072 24051 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bf4f108cfee4559a9f82f95024ff33d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:15 UTC) from {username='oryx', principal='oryx/127.15.217.125@KRBTEST.COM'} at 127.0.0.1:37908
I20260810 07:15:15.005344 24415 tablet_replica.cc:330] T 2bf4f108cfee4559a9f82f95024ff33d P 87f6de6d8282454f9da57f1a17b89e6e: stopping tablet replica
I20260810 07:15:15.005417 24415 raft_consensus.cc:2229] T 2bf4f108cfee4559a9f82f95024ff33d P 87f6de6d8282454f9da57f1a17b89e6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:15.005470 24415 raft_consensus.cc:2258] T 2bf4f108cfee4559a9f82f95024ff33d P 87f6de6d8282454f9da57f1a17b89e6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:15.005492 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 23728
I20260810 07:15:15.005967 24415 ts_tablet_manager.cc:1905] T 2bf4f108cfee4559a9f82f95024ff33d P 87f6de6d8282454f9da57f1a17b89e6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:15.006796 24412 tablet_replica.cc:330] T 2bf4f108cfee4559a9f82f95024ff33d P ceed6041617a45b6ab60d5fb2064d12d: stopping tablet replica
I20260810 07:15:15.006851 24412 raft_consensus.cc:2229] T 2bf4f108cfee4559a9f82f95024ff33d P ceed6041617a45b6ab60d5fb2064d12d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:15.006887 24412 raft_consensus.cc:2258] T 2bf4f108cfee4559a9f82f95024ff33d P ceed6041617a45b6ab60d5fb2064d12d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:15.007285 24412 ts_tablet_manager.cc:1905] T 2bf4f108cfee4559a9f82f95024ff33d P ceed6041617a45b6ab60d5fb2064d12d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:15.008975 24412 ts_tablet_manager.cc:1918] T 2bf4f108cfee4559a9f82f95024ff33d P ceed6041617a45b6ab60d5fb2064d12d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:15:15.009063 24412 log.cc:1196] T 2bf4f108cfee4559a9f82f95024ff33d P ceed6041617a45b6ab60d5fb2064d12d: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-2/wal/wals/2bf4f108cfee4559a9f82f95024ff33d
I20260810 07:15:15.009130 24415 ts_tablet_manager.cc:1918] T 2bf4f108cfee4559a9f82f95024ff33d P 87f6de6d8282454f9da57f1a17b89e6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:15:15.009209 24415 log.cc:1196] T 2bf4f108cfee4559a9f82f95024ff33d P 87f6de6d8282454f9da57f1a17b89e6e: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346081759552-16229-0/minicluster-data/ts-1/wal/wals/2bf4f108cfee4559a9f82f95024ff33d
I20260810 07:15:15.009348 24412 ts_tablet_manager.cc:1939] T 2bf4f108cfee4559a9f82f95024ff33d P ceed6041617a45b6ab60d5fb2064d12d: Deleting consensus metadata
I20260810 07:15:15.009523 24415 ts_tablet_manager.cc:1939] T 2bf4f108cfee4559a9f82f95024ff33d P 87f6de6d8282454f9da57f1a17b89e6e: Deleting consensus metadata
I20260810 07:15:15.009727 23546 catalog_manager.cc:4841] TS ceed6041617a45b6ab60d5fb2064d12d (127.15.217.67:39201): tablet 2bf4f108cfee4559a9f82f95024ff33d (table test-table [id=d85a7239865e4da786e99f8e3ef41b3f]) successfully deleted
I20260810 07:15:15.009845 23545 catalog_manager.cc:4841] TS 87f6de6d8282454f9da57f1a17b89e6e (127.15.217.66:46239): tablet 2bf4f108cfee4559a9f82f95024ff33d (table test-table [id=d85a7239865e4da786e99f8e3ef41b3f]) successfully deleted
I20260810 07:15:15.019265 24357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:15.019098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:34201 (local address 127.0.0.1:34500)
0810 07:15:15.019171 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:15.019209 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.65:34201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 07:15:15.019452 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 23983
W20260810 07:15:15.019472 23546 catalog_manager.cc:4584] TS 2a7624fba8924223967850a43cc913c2 (127.15.217.65:34201): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2bf4f108cfee4559a9f82f95024ff33d: Network error: Client connection negotiation failed: client connection to 127.15.217.65:34201: connect: Connection refused (error 111)
I20260810 07:15:15.029748 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 24134
I20260810 07:15:15.037084 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 23414
I20260810 07:15:15.044956 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 23622
2026-08-10T07:15:15Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (4998 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:15:15 dist-test-slave-10pc krb5kdc[24420](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:15:15 dist-test-slave-10pc krb5kdc[24420](info): set up 2 sockets
Aug 10 07:15:15 dist-test-slave-10pc krb5kdc[24420](info): commencing operation
krb5kdc: starting...
W20260810 07:15:16.152374 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:15:16 dist-test-slave-10pc krb5kdc[24420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:15:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:15:16Z Disabled control of system clock
WARNING: no policy specified for oryx/127.15.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.126.keytab.
Entry for principal oryx/127.15.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.126.keytab.
I20260810 07:15:16.285332 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:33813
--webserver_interface=127.15.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.126.keytab
--principal=oryx/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:32905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:33813
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:16.373176 24434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:16.373387 24434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:16.373411 24434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:16.374614 24434 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:16.374661 24434 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 07:15:16.374675 24434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:16.374688 24434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:16.374747 24434 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:16.374763 24434 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:15:16.376286 24434 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:32905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:33813
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.126.keytab
--principal=oryx/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:33813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:16.376590 24434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:16.376860 24434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:16.382053 24439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:16.382020 24443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:16.382074 24441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:16.382261 24434 server_base.cc:1034] running on GCE node
I20260810 07:15:16.382613 24434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:16.382882 24434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:16.384037 24434 hybrid_clock.cc:648] HybridClock initialized: now 1786346116384012 us; error 43 us; skew 500 ppm
Aug 10 07:15:16 dist-test-slave-10pc krb5kdc[24420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346116, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:16.386843 24434 init.cc:377] Logged in from keytab as oryx/127.15.217.126@KRBTEST.COM (short username oryx)
I20260810 07:15:16.387327 24434 webserver.cc:460] Webserver started at http://127.15.217.126:38053/ using document root <none> and password file <none>
I20260810 07:15:16.387564 24434 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:16.387634 24434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:16.387792 24434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:16.388989 24434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "0b336a4b1aac4e909a20ccd2a426c4a7"
format_stamp: "Formatted at 2026-08-10 07:15:16 on dist-test-slave-10pc"
server_key: "abc1208a96211a75cfcf114cf5d70b1c"
server_key_iv: "9b2489f8b537bf0e7d7212a71efd608a"
server_key_version: "encryptionkey@0"
I20260810 07:15:16.389400 24434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "0b336a4b1aac4e909a20ccd2a426c4a7"
format_stamp: "Formatted at 2026-08-10 07:15:16 on dist-test-slave-10pc"
server_key: "abc1208a96211a75cfcf114cf5d70b1c"
server_key_iv: "9b2489f8b537bf0e7d7212a71efd608a"
server_key_version: "encryptionkey@0"
I20260810 07:15:16.390857 24434 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:15:16.391710 24450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:16.391908 24434 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:16.392022 24434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "0b336a4b1aac4e909a20ccd2a426c4a7"
format_stamp: "Formatted at 2026-08-10 07:15:16 on dist-test-slave-10pc"
server_key: "abc1208a96211a75cfcf114cf5d70b1c"
server_key_iv: "9b2489f8b537bf0e7d7212a71efd608a"
server_key_version: "encryptionkey@0"
I20260810 07:15:16.392103 24434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:16.405432 24434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:16.407997 24434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:16.408190 24434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:16.413419 24434 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:33813
I20260810 07:15:16.413476 24519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:33813 every 8 connection(s)
I20260810 07:15:16.413823 24434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:15:16.414495 24520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:16.416976 24520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b336a4b1aac4e909a20ccd2a426c4a7: Bootstrap starting.
I20260810 07:15:16.417675 24520 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b336a4b1aac4e909a20ccd2a426c4a7: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:16.417999 24520 log.cc:824] T 00000000000000000000000000000000 P 0b336a4b1aac4e909a20ccd2a426c4a7: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:16.418716 24520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b336a4b1aac4e909a20ccd2a426c4a7: No bootstrap required, opened a new log
I20260810 07:15:16.420871 24520 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b336a4b1aac4e909a20ccd2a426c4a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b336a4b1aac4e909a20ccd2a426c4a7" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 33813 } }
I20260810 07:15:16.421022 24520 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b336a4b1aac4e909a20ccd2a426c4a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:16.421046 24520 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b336a4b1aac4e909a20ccd2a426c4a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b336a4b1aac4e909a20ccd2a426c4a7, State: Initialized, Role: FOLLOWER
I20260810 07:15:16.421231 24520 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b336a4b1aac4e909a20ccd2a426c4a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b336a4b1aac4e909a20ccd2a426c4a7" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 33813 } }
I20260810 07:15:16.421299 24520 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b336a4b1aac4e909a20ccd2a426c4a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:15:16.421343 24520 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b336a4b1aac4e909a20ccd2a426c4a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:15:16.421392 24520 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b336a4b1aac4e909a20ccd2a426c4a7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:16.421939 24520 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b336a4b1aac4e909a20ccd2a426c4a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b336a4b1aac4e909a20ccd2a426c4a7" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 33813 } }
I20260810 07:15:16.422029 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 24434
I20260810 07:15:16.422063 24520 leader_election.cc:302] T 00000000000000000000000000000000 P 0b336a4b1aac4e909a20ccd2a426c4a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b336a4b1aac4e909a20ccd2a426c4a7; no voters:
I20260810 07:15:16.422152 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:15:16.422259 24520 leader_election.cc:288] T 00000000000000000000000000000000 P 0b336a4b1aac4e909a20ccd2a426c4a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:15:16.422309 24524 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b336a4b1aac4e909a20ccd2a426c4a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:16.422405 16229 external_mini_cluster.cc:1428] Setting key 81eb0aa0bc0b305fe5e53b66dffd2136
I20260810 07:15:16.422475 24524 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b336a4b1aac4e909a20ccd2a426c4a7 [term 1 LEADER]: Becoming Leader. State: Replica: 0b336a4b1aac4e909a20ccd2a426c4a7, State: Running, Role: LEADER
I20260810 07:15:16.422569 24520 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b336a4b1aac4e909a20ccd2a426c4a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:16.422585 24524 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b336a4b1aac4e909a20ccd2a426c4a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b336a4b1aac4e909a20ccd2a426c4a7" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 33813 } }
I20260810 07:15:16.422966 24524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b336a4b1aac4e909a20ccd2a426c4a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b336a4b1aac4e909a20ccd2a426c4a7. Latest consensus state: current_term: 1 leader_uuid: "0b336a4b1aac4e909a20ccd2a426c4a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b336a4b1aac4e909a20ccd2a426c4a7" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 33813 } } }
I20260810 07:15:16.423067 24524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b336a4b1aac4e909a20ccd2a426c4a7 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:16.422976 24525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b336a4b1aac4e909a20ccd2a426c4a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b336a4b1aac4e909a20ccd2a426c4a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b336a4b1aac4e909a20ccd2a426c4a7" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 33813 } } }
I20260810 07:15:16.423270 24525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b336a4b1aac4e909a20ccd2a426c4a7 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:16.423599 24535 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:16.424191 24535 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:15:16.426093 24535 catalog_manager.cc:1347] Generated new cluster ID: 4b6851f08d4e45e7931b763ac00985cd
I20260810 07:15:16.426152 24535 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 07:15:16 dist-test-slave-10pc krb5kdc[24420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM
I20260810 07:15:16.433526 24537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:16.423166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41234 (local address 127.15.217.126:33813)
0810 07:15:16.423323 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:16.423334 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:16.423439 (+ 105us) server_negotiation.cc:407] Connection header received
0810 07:15:16.423632 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:16.423640 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:16.423722 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:16.423861 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:16.424877 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:16.426194 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:16.426986 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:16.427466 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:16.427587 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:16.430043 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:16.430047 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:16.430048 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:16.430061 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:16.431808 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:16.432209 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:16.432212 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:16.432213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:16.432320 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:16.432563 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:16.432564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:16.432564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:16.432821 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:16.432996 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:16.433106 (+ 110us) server_negotiation.cc:299] Negotiation successful
0810 07:15:16.433352 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":75,"threads_started":1}
I20260810 07:15:16.447856 24535 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:15:16.448510 24535 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:16.461144 24535 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b336a4b1aac4e909a20ccd2a426c4a7: Generated new TSK 0
I20260810 07:15:16.461409 24535 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.15.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.65.keytab.
Entry for principal oryx/127.15.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.65.keytab.
I20260810 07:15:16.491142 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.65:0
--local_ip_for_outbound_sockets=127.15.217.65
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:33813
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.65.keytab
--principal=oryx/127.15.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:32905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:16.579514 24550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:16.579732 24550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:16.579759 24550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:16.580993 24550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:16.581040 24550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:16.581061 24550 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 07:15:16.581079 24550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.65
I20260810 07:15:16.582717 24550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:32905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.65.keytab
--principal=oryx/127.15.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.217.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.217.126:33813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.65
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:16.582996 24550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:16.583274 24550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:16.588776 24559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:16.588783 24556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:16.588835 24550 server_base.cc:1034] running on GCE node
W20260810 07:15:16.588783 24557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:16.589157 24550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:16.589460 24550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:16.590636 24550 hybrid_clock.cc:648] HybridClock initialized: now 1786346116590624 us; error 33 us; skew 500 ppm
Aug 10 07:15:16 dist-test-slave-10pc krb5kdc[24420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346116, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:16.593549 24550 init.cc:377] Logged in from keytab as oryx/127.15.217.65@KRBTEST.COM (short username oryx)
I20260810 07:15:16.594014 24550 webserver.cc:460] Webserver started at http://127.15.217.65:33869/ using document root <none> and password file <none>
I20260810 07:15:16.594291 24550 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:16.594364 24550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:16.594516 24550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:16.595909 24550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-0/data/instance:
uuid: "dc01a69a8f784435b348edba6864ce55"
format_stamp: "Formatted at 2026-08-10 07:15:16 on dist-test-slave-10pc"
server_key: "741aa71a96a026030b584113712318e7"
server_key_iv: "0ee1178062c9c3b27b2919af6d9d03de"
server_key_version: "encryptionkey@0"
I20260810 07:15:16.596443 24550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance:
uuid: "dc01a69a8f784435b348edba6864ce55"
format_stamp: "Formatted at 2026-08-10 07:15:16 on dist-test-slave-10pc"
server_key: "741aa71a96a026030b584113712318e7"
server_key_iv: "0ee1178062c9c3b27b2919af6d9d03de"
server_key_version: "encryptionkey@0"
I20260810 07:15:16.598019 24550 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:15:16.598850 24569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:16.599035 24550 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:15:16.599143 24550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-0/wal
uuid: "dc01a69a8f784435b348edba6864ce55"
format_stamp: "Formatted at 2026-08-10 07:15:16 on dist-test-slave-10pc"
server_key: "741aa71a96a026030b584113712318e7"
server_key_iv: "0ee1178062c9c3b27b2919af6d9d03de"
server_key_version: "encryptionkey@0"
I20260810 07:15:16.599243 24550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:16.616156 24550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:16.618921 24550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:16.619119 24550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:16.619743 24550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:16.619814 24550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:16.619865 24550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:16.619892 24550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:16.627816 24550 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.65:41485
Aug 10 07:15:16 dist-test-slave-10pc krb5kdc[24420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346116, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM
I20260810 07:15:16.627919 24692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.65:41485 every 8 connection(s)
I20260810 07:15:16.628291 24550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
I20260810 07:15:16.631124 24537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:16.620439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:39919 (local address 127.15.217.126:33813)
0810 07:15:16.620533 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:16.620536 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:16.620777 (+ 241us) server_negotiation.cc:407] Connection header received
0810 07:15:16.621350 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:16.621352 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:16.621374 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:16.621459 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:16.622416 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:16.623304 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:16.624701 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:16.625122 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:16.625206 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:16.628543 (+ 3337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:16.628546 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:16.628547 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:16.628559 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:16.629905 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:16.630295 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:16.630296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:16.630296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:16.630352 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:16.630578 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:16.630578 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:16.630578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:16.630688 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:16.630739 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:16.630984 (+ 245us) server_negotiation.cc:299] Negotiation successful
0810 07:15:16.631058 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 07:15:16.631345 24578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:16.620442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:33813 (local address 127.15.217.65:39919)
0810 07:15:16.620647 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:16.620687 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:16.621223 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:16.621539 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:16.621543 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:16.621959 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:16.622289 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:16.622298 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:16.623415 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:16.623417 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:16.624598 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:16.624603 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:16.625277 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:16.625279 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:16.625492 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:16.626190 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:16.626200 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:16.628372 (+ 2172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:16.629979 (+ 1607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:16.629986 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:16.629988 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:16.630223 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:16.630415 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:16.630415 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:16.630416 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:16.630522 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:16.630746 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:16.630750 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:16.630871 (+ 121us) client_negotiation.cc:769] Sending connection context
0810 07:15:16.630955 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 07:15:16.631088 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":86,"threads_started":1}
I20260810 07:15:16.631958 24693 heartbeater.cc:347] Connected to a master server at 127.15.217.126:33813
I20260810 07:15:16.632081 24693 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:16.632306 24693 heartbeater.cc:511] Master 127.15.217.126:33813 requested a full tablet report, sending...
I20260810 07:15:16.632997 24470 ts_manager.cc:194] Registered new tserver with Master: dc01a69a8f784435b348edba6864ce55 (127.15.217.65:41485)
I20260810 07:15:16.633651 24470 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.217.65@KRBTEST.COM'} at 127.15.217.65:39919
I20260810 07:15:16.637670 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 24550
I20260810 07:15:16.637805 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance
I20260810 07:15:16.638087 16229 external_mini_cluster.cc:1428] Setting key 5e308d30bc8a0c2921726b395b0932cd
I20260810 07:15:16.639323 24537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:16.633182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:38261 (local address 127.15.217.126:33813)
0810 07:15:16.633232 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:16.633235 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:16.633337 (+ 102us) server_negotiation.cc:407] Connection header received
0810 07:15:16.633396 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:16.633397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:16.633407 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:16.633461 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:16.633810 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:16.634472 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:16.635164 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:16.635349 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:16.635403 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:16.636347 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:16.636348 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:16.636348 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:16.636353 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:16.637886 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:16.638370 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:16.638372 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:16.638372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:16.638461 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:16.638712 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:16.638713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:16.638714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:16.638867 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:16.638936 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:16.639133 (+ 197us) server_negotiation.cc:299] Negotiation successful
0810 07:15:16.639221 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 07:15:16.639330 24699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:16.633130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:33813 (local address 127.15.217.65:38261)
0810 07:15:16.633266 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:16.633276 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:16.633336 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:16.633469 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:16.633470 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:16.633665 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:16.633721 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:16.633723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:16.634574 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:16.634575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:16.635086 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:16.635088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:16.635419 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:16.635420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:16.635475 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:16.635958 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:16.635964 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:16.636266 (+ 302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:16.638053 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:16.638054 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:16.638054 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:16.638276 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:16.638562 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:16.638563 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:16.638563 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:16.638626 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:16.638939 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:16.638948 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:16.639011 (+ 63us) client_negotiation.cc:769] Sending connection context
0810 07:15:16.639091 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 07:15:16.639219 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":68,"threads_started":1}
WARNING: no policy specified for oryx/127.15.217.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.217.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.66.keytab.
Entry for principal oryx/127.15.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.66.keytab.
I20260810 07:15:16.667953 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.66:0
--local_ip_for_outbound_sockets=127.15.217.66
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:33813
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.66.keytab
--principal=oryx/127.15.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:32905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:16.754648 24702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:16.754925 24702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:16.754971 24702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:16.756207 24702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:16.756283 24702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:16.756317 24702 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 07:15:16.756356 24702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.66
I20260810 07:15:16.757989 24702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:32905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.66.keytab
--principal=oryx/127.15.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.217.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.217.126:33813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.66
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:16.758299 24702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:16.758565 24702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:16.763883 24707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:16.763883 24711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:16.763923 24708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:16.764129 24702 server_base.cc:1034] running on GCE node
I20260810 07:15:16.764487 24702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:16.764761 24702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:16.765916 24702 hybrid_clock.cc:648] HybridClock initialized: now 1786346116765907 us; error 31 us; skew 500 ppm
Aug 10 07:15:16 dist-test-slave-10pc krb5kdc[24420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346116, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:16.768757 24702 init.cc:377] Logged in from keytab as oryx/127.15.217.66@KRBTEST.COM (short username oryx)
I20260810 07:15:16.769152 24702 webserver.cc:460] Webserver started at http://127.15.217.66:40745/ using document root <none> and password file <none>
I20260810 07:15:16.769379 24702 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:16.769423 24702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:16.769516 24702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:16.770556 24702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-1/data/instance:
uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d"
format_stamp: "Formatted at 2026-08-10 07:15:16 on dist-test-slave-10pc"
server_key: "ae904a75184e5dea08088a9ce9426f7b"
server_key_iv: "fc51ddcc332625a4384c2502eacf954f"
server_key_version: "encryptionkey@0"
I20260810 07:15:16.770874 24702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance:
uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d"
format_stamp: "Formatted at 2026-08-10 07:15:16 on dist-test-slave-10pc"
server_key: "ae904a75184e5dea08088a9ce9426f7b"
server_key_iv: "fc51ddcc332625a4384c2502eacf954f"
server_key_version: "encryptionkey@0"
I20260810 07:15:16.772248 24702 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:15:16.773069 24719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:16.773234 24702 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 07:15:16.773324 24702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-1/wal
uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d"
format_stamp: "Formatted at 2026-08-10 07:15:16 on dist-test-slave-10pc"
server_key: "ae904a75184e5dea08088a9ce9426f7b"
server_key_iv: "fc51ddcc332625a4384c2502eacf954f"
server_key_version: "encryptionkey@0"
I20260810 07:15:16.773383 24702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:16.796886 24702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:16.799031 24702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:16.799158 24702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:16.799682 24702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:16.799755 24702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:16.799800 24702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:16.799829 24702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 07:15:16 dist-test-slave-10pc krb5kdc[24420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346116, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM
I20260810 07:15:16.807428 24702 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.66:46469
I20260810 07:15:16.807523 24844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.66:46469 every 8 connection(s)
I20260810 07:15:16.807879 24702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
I20260810 07:15:16.810389 24537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:16.800213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:48011 (local address 127.15.217.126:33813)
0810 07:15:16.800277 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:16.800280 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:16.800514 (+ 234us) server_negotiation.cc:407] Connection header received
0810 07:15:16.801115 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:16.801117 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:16.801132 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:16.801223 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:16.802050 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:16.802933 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:16.804693 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:16.804911 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:16.805015 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:16.807313 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:16.807315 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:16.807316 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:16.807324 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:16.808910 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:16.809339 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:16.809341 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:16.809341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:16.809407 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:16.809686 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:16.809687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:16.809688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:16.809819 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:16.809901 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:16.810207 (+ 306us) server_negotiation.cc:299] Negotiation successful
0810 07:15:16.810295 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 07:15:16.810578 24728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:16.800195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:33813 (local address 127.15.217.66:48011)
0810 07:15:16.800363 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:16.800406 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:16.800994 (+ 588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:16.801348 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:16.801352 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:16.801737 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:16.801943 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:16.801949 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:16.803060 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:16.803061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:16.804590 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:16.804594 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:16.805043 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:16.805045 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:16.805183 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:16.805733 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:16.805742 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:16.807196 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:16.809002 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:16.809011 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:16.809013 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:16.809254 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:16.809479 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:16.809480 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:16.809480 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:16.809602 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:16.809893 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:16.809900 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:16.810058 (+ 158us) client_negotiation.cc:769] Sending connection context
0810 07:15:16.810174 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 07:15:16.810348 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":86,"threads_started":1}
I20260810 07:15:16.811090 24845 heartbeater.cc:347] Connected to a master server at 127.15.217.126:33813
I20260810 07:15:16.811180 24845 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:16.811383 24845 heartbeater.cc:511] Master 127.15.217.126:33813 requested a full tablet report, sending...
I20260810 07:15:16.811851 24469 ts_manager.cc:194] Registered new tserver with Master: 0b2b56cd5f6d4f64be7f73b6a75c439d (127.15.217.66:46469)
I20260810 07:15:16.812345 24469 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.217.66@KRBTEST.COM'} at 127.15.217.66:48011
I20260810 07:15:16.814497 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 24702
I20260810 07:15:16.814628 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance
I20260810 07:15:16.814842 16229 external_mini_cluster.cc:1428] Setting key 84ba605f326477c02222a0b6c3684551
I20260810 07:15:16.818153 24537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:16.812289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:42171 (local address 127.15.217.126:33813)
0810 07:15:16.812336 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:16.812338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:16.812448 (+ 110us) server_negotiation.cc:407] Connection header received
0810 07:15:16.812519 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:16.812520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:16.812530 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:16.812592 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:16.812917 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:16.813397 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:16.814098 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:16.814307 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:16.814383 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:16.815420 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:16.815421 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:16.815422 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:16.815428 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:16.816926 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:16.817313 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:16.817314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:16.817314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:16.817373 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:16.817599 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:16.817600 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:16.817601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:16.817755 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:16.817814 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:16.818029 (+ 215us) server_negotiation.cc:299] Negotiation successful
0810 07:15:16.818095 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 07:15:16.818234 24852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:16.812221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:33813 (local address 127.15.217.66:42171)
0810 07:15:16.812362 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:16.812373 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:16.812441 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:16.812595 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:16.812596 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:16.812765 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:16.812834 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:16.812836 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:16.813500 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:16.813501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:16.814010 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:16.814012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:16.814397 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:16.814398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:16.814469 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:16.814936 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:16.814941 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:16.815325 (+ 384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:16.817048 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:16.817049 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:16.817050 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:16.817234 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:16.817450 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:16.817451 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:16.817452 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:16.817527 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:16.817826 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:16.817827 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:16.817881 (+ 54us) client_negotiation.cc:769] Sending connection context
0810 07:15:16.817957 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 07:15:16.818082 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":76,"threads_started":1}
WARNING: no policy specified for oryx/127.15.217.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.217.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.67.keytab.
Entry for principal oryx/127.15.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.67.keytab.
I20260810 07:15:16.843183 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.67:0
--local_ip_for_outbound_sockets=127.15.217.67
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:33813
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.67.keytab
--principal=oryx/127.15.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:32905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:16.930279 24855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:16.930503 24855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:16.930527 24855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:16.931697 24855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:16.931736 24855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:16.931778 24855 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 07:15:16.931813 24855 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.67
I20260810 07:15:16.933503 24855 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:32905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.67.keytab
--principal=oryx/127.15.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.217.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.217.126:33813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.67
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:16.933775 24855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:16.934020 24855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:16.939134 24865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:16.939167 24862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:16.939137 24863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:16.939410 24855 server_base.cc:1034] running on GCE node
I20260810 07:15:16.939633 24855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:16.939885 24855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:16.941046 24855 hybrid_clock.cc:648] HybridClock initialized: now 1786346116941031 us; error 29 us; skew 500 ppm
Aug 10 07:15:16 dist-test-slave-10pc krb5kdc[24420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346116, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:16.943698 24855 init.cc:377] Logged in from keytab as oryx/127.15.217.67@KRBTEST.COM (short username oryx)
I20260810 07:15:16.944156 24855 webserver.cc:460] Webserver started at http://127.15.217.67:33609/ using document root <none> and password file <none>
I20260810 07:15:16.944408 24855 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:16.944509 24855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:16.944679 24855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:16.945828 24855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-2/data/instance:
uuid: "da72d89bb77b471bbbdd19123c4776a9"
format_stamp: "Formatted at 2026-08-10 07:15:16 on dist-test-slave-10pc"
server_key: "913ce1a81b586a4c6c40eba5a9a7eeb9"
server_key_iv: "e354abe8b59fbcd7e298c820a658b737"
server_key_version: "encryptionkey@0"
I20260810 07:15:16.946234 24855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance:
uuid: "da72d89bb77b471bbbdd19123c4776a9"
format_stamp: "Formatted at 2026-08-10 07:15:16 on dist-test-slave-10pc"
server_key: "913ce1a81b586a4c6c40eba5a9a7eeb9"
server_key_iv: "e354abe8b59fbcd7e298c820a658b737"
server_key_version: "encryptionkey@0"
I20260810 07:15:16.947687 24855 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:15:16.948431 24873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:16.948649 24855 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:16.948755 24855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-2/wal
uuid: "da72d89bb77b471bbbdd19123c4776a9"
format_stamp: "Formatted at 2026-08-10 07:15:16 on dist-test-slave-10pc"
server_key: "913ce1a81b586a4c6c40eba5a9a7eeb9"
server_key_iv: "e354abe8b59fbcd7e298c820a658b737"
server_key_version: "encryptionkey@0"
I20260810 07:15:16.948843 24855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:16.963014 24855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:16.965317 24855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:16.965495 24855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:16.966049 24855 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:16.966120 24855 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:16.966176 24855 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:16.966207 24855 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:16.973593 24855 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.67:40883
Aug 10 07:15:16 dist-test-slave-10pc krb5kdc[24420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346116, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM
I20260810 07:15:16.973676 25001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.67:40883 every 8 connection(s)
I20260810 07:15:16.974009 24855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
I20260810 07:15:16.977039 24537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:16.966595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:42959 (local address 127.15.217.126:33813)
0810 07:15:16.966661 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:16.966663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:16.966915 (+ 252us) server_negotiation.cc:407] Connection header received
0810 07:15:16.967586 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:16.967588 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:16.967606 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:16.967706 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:16.968543 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:16.969393 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:16.971054 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:16.971412 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:16.971498 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:16.974260 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:16.974262 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:16.974262 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:16.974272 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:16.975607 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:16.976100 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:16.976101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:16.976102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:16.976168 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:16.976412 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:16.976412 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:16.976413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:16.976588 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:16.976645 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:16.976893 (+ 248us) server_negotiation.cc:299] Negotiation successful
0810 07:15:16.976970 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 07:15:16.977360 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:16.966598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:33813 (local address 127.15.217.67:42959)
0810 07:15:16.966789 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:16.966828 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:16.967424 (+ 596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:16.967792 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:16.967797 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:16.968203 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:16.968406 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:16.968411 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:16.969489 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:16.969491 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:16.970920 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:16.970924 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:16.971516 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:16.971517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:16.971647 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:16.972238 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:16.972247 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:16.974145 (+ 1898us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:16.975683 (+ 1538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:16.975690 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:16.975693 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:16.975929 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:16.976240 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:16.976241 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:16.976241 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:16.976351 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:16.976636 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:16.976640 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:16.976765 (+ 125us) client_negotiation.cc:769] Sending connection context
0810 07:15:16.976852 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 07:15:16.977055 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":106,"threads_started":1}
I20260810 07:15:16.977922 25002 heartbeater.cc:347] Connected to a master server at 127.15.217.126:33813
I20260810 07:15:16.978021 25002 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:16.978233 25002 heartbeater.cc:511] Master 127.15.217.126:33813 requested a full tablet report, sending...
I20260810 07:15:16.978695 24469 ts_manager.cc:194] Registered new tserver with Master: da72d89bb77b471bbbdd19123c4776a9 (127.15.217.67:40883)
I20260810 07:15:16.979106 24469 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.217.67@KRBTEST.COM'} at 127.15.217.67:42959
I20260810 07:15:16.979691 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 24855
I20260810 07:15:16.979837 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance
I20260810 07:15:16.980175 16229 external_mini_cluster.cc:1428] Setting key bb16cb8231724066466ac18f838dc493
I20260810 07:15:16.981142 16229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:15:16.986279 25008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:16.979140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:33813 (local address 127.15.217.67:40419)
0810 07:15:16.979348 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:16.979359 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:16.979457 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:16.979715 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:16.979716 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:16.979983 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:16.980069 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:16.980072 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:16.980939 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:16.980940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:16.981832 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:16.981836 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:16.982190 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:16.982191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:16.982274 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:16.982739 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:16.982746 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:16.983241 (+ 495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:16.985023 (+ 1782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:16.985024 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:16.985025 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:16.985266 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:16.985518 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:16.985519 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:16.985520 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:16.985596 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:16.985916 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:16.985917 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:16.986015 (+ 98us) client_negotiation.cc:769] Sending connection context
0810 07:15:16.986079 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 07:15:16.986183 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":71,"threads_started":1}
I20260810 07:15:16.986370 24537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:16.979215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:40419 (local address 127.15.217.126:33813)
0810 07:15:16.979261 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:16.979263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:16.979430 (+ 167us) server_negotiation.cc:407] Connection header received
0810 07:15:16.979572 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:16.979573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:16.979586 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:16.979659 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:16.980186 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:16.980849 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:16.981921 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:16.982109 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:16.982172 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:16.983337 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:16.983339 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:16.983339 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:16.983345 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:16.984879 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:16.985351 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:16.985352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:16.985353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:16.985421 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:16.985687 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:16.985688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:16.985688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:16.985845 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:16.985917 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:16.986128 (+ 211us) server_negotiation.cc:299] Negotiation successful
0810 07:15:16.986230 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
Aug 10 07:15:16 dist-test-slave-10pc krb5kdc[24420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346116, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 07:15:17 dist-test-slave-10pc krb5kdc[24420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:17 dist-test-slave-10pc krb5kdc[24420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
W20260810 07:15:17.006775 24537 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:17.000942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41246 (local address 127.15.217.126:33813)
0810 07:15:17.001012 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:17.001014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:17.001024 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:15:17.001108 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:17.001108 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:17.001118 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:17.001177 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:17.001764 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.002278 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.003061 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.003249 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.003305 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:17.006652 (+ 3347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41246: BlockingRecv error: recv got EOF from 127.0.0.1:41246 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 10 07:15:17 dist-test-slave-10pc krb5kdc[24420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346116, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM
I20260810 07:15:17.015163 24537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:17.007652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41256 (local address 127.15.217.126:33813)
0810 07:15:17.007729 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:17.007731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:17.007816 (+ 85us) server_negotiation.cc:407] Connection header received
0810 07:15:17.007901 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:17.007902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:17.007913 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:17.007984 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:17.008530 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.009014 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.009729 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.009924 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.010024 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:17.012433 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:17.012435 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:17.012435 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:17.012444 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:17.013907 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:17.014307 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:17.014308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:17.014309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:17.014374 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:17.014588 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:17.014589 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:17.014589 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:17.014733 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:17.014849 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:17.014954 (+ 105us) server_negotiation.cc:299] Negotiation successful
0810 07:15:17.015049 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:15:17.017242 24469 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41256:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:15:17.018849 24469 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:15:17.025712 25025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:17.021321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:41485 (local address 127.0.0.1:57402)
0810 07:15:17.021494 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:17.021537 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:17.021608 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:17.022158 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:17.022162 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:17.022177 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:17.022271 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:17.022274 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.023359 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.023360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:17.024298 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:17.024301 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.025391 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.025392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:17.025479 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:17.025495 (+ 16us) client_negotiation.cc:769] Sending connection context
0810 07:15:17.025573 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 07:15:17.025623 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":2,"thread_start_us":89,"threads_started":1}
I20260810 07:15:17.025760 25030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:17.021742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57402 (local address 127.15.217.65:41485)
0810 07:15:17.021909 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:17.021913 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:17.021925 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:15:17.021974 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:17.021976 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:17.021995 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:17.022076 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:17.022364 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.023269 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.024398 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.025302 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.025381 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:17.025461 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:17.025605 (+ 144us) server_negotiation.cc:299] Negotiation successful
0810 07:15:17.025650 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":114,"threads_started":1}
I20260810 07:15:17.026268 25028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:17.021533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55606 (local address 127.15.217.67:40883)
0810 07:15:17.021837 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:17.021842 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:17.021861 (+ 19us) server_negotiation.cc:407] Connection header received
0810 07:15:17.021904 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:17.021906 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:17.021922 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:17.022024 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:17.022419 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.023533 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.024967 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.025727 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.025801 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:17.025883 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:17.026113 (+ 230us) server_negotiation.cc:299] Negotiation successful
0810 07:15:17.026155 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":79,"threads_started":1}
I20260810 07:15:17.026691 25027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:17.021337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:46469 (local address 127.0.0.1:49630)
0810 07:15:17.021956 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:17.021967 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:17.022062 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:17.022339 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:17.022340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:17.022342 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:17.022374 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:17.022375 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.023753 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.023754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:17.025127 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:17.025130 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.026431 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.026432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:17.026510 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:17.026515 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:15:17.026574 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 07:15:17.026624 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":92,"threads_started":1}
I20260810 07:15:17.026767 25029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:17.021585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49630 (local address 127.15.217.66:46469)
0810 07:15:17.021801 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:17.021807 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:17.022033 (+ 226us) server_negotiation.cc:407] Connection header received
0810 07:15:17.022163 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:17.022165 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:17.022182 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:17.022236 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:17.022478 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.023670 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.025213 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.026241 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.026315 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:17.026396 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:17.026612 (+ 216us) server_negotiation.cc:299] Negotiation successful
0810 07:15:17.026657 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":109,"threads_started":1}
I20260810 07:15:17.026753 24622 tablet_service.cc:1467] Processing CreateTablet for tablet 74aa62dcf3624cfb853be44b6121dc68 (DEFAULT_TABLE table=test-table [id=cb2460f62af946e388706a0ab1441478]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:17.027238 24622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74aa62dcf3624cfb853be44b6121dc68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:17.027477 24774 tablet_service.cc:1467] Processing CreateTablet for tablet 74aa62dcf3624cfb853be44b6121dc68 (DEFAULT_TABLE table=test-table [id=cb2460f62af946e388706a0ab1441478]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:17.027860 24774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74aa62dcf3624cfb853be44b6121dc68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:17.026206 25026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:17.021321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:40883 (local address 127.0.0.1:55606)
0810 07:15:17.021721 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:17.021731 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:17.021770 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:17.022160 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:17.022163 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:17.022166 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:17.022269 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:17.022273 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.023634 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.023635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:17.024857 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:17.024861 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.025949 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.025950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:17.026032 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:17.026038 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 07:15:17.026093 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 07:15:17.026135 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"mutex_wait_us":33,"thread_start_us":59,"threads_started":1}
I20260810 07:15:17.029273 24932 tablet_service.cc:1467] Processing CreateTablet for tablet 74aa62dcf3624cfb853be44b6121dc68 (DEFAULT_TABLE table=test-table [id=cb2460f62af946e388706a0ab1441478]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:17.029697 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74aa62dcf3624cfb853be44b6121dc68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:17.032212 25032 tablet_bootstrap.cc:492] T 74aa62dcf3624cfb853be44b6121dc68 P 0b2b56cd5f6d4f64be7f73b6a75c439d: Bootstrap starting.
I20260810 07:15:17.032823 25031 tablet_bootstrap.cc:492] T 74aa62dcf3624cfb853be44b6121dc68 P dc01a69a8f784435b348edba6864ce55: Bootstrap starting.
I20260810 07:15:17.033480 25031 tablet_bootstrap.cc:654] T 74aa62dcf3624cfb853be44b6121dc68 P dc01a69a8f784435b348edba6864ce55: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:17.033735 25031 log.cc:824] T 74aa62dcf3624cfb853be44b6121dc68 P dc01a69a8f784435b348edba6864ce55: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:17.034250 25032 tablet_bootstrap.cc:654] T 74aa62dcf3624cfb853be44b6121dc68 P 0b2b56cd5f6d4f64be7f73b6a75c439d: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:17.034469 25031 tablet_bootstrap.cc:492] T 74aa62dcf3624cfb853be44b6121dc68 P dc01a69a8f784435b348edba6864ce55: No bootstrap required, opened a new log
I20260810 07:15:17.034540 25031 ts_tablet_manager.cc:1397] T 74aa62dcf3624cfb853be44b6121dc68 P dc01a69a8f784435b348edba6864ce55: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260810 07:15:17.034617 25032 log.cc:824] T 74aa62dcf3624cfb853be44b6121dc68 P 0b2b56cd5f6d4f64be7f73b6a75c439d: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:17.034989 25033 tablet_bootstrap.cc:492] T 74aa62dcf3624cfb853be44b6121dc68 P da72d89bb77b471bbbdd19123c4776a9: Bootstrap starting.
I20260810 07:15:17.035414 25032 tablet_bootstrap.cc:492] T 74aa62dcf3624cfb853be44b6121dc68 P 0b2b56cd5f6d4f64be7f73b6a75c439d: No bootstrap required, opened a new log
I20260810 07:15:17.035490 25032 ts_tablet_manager.cc:1397] T 74aa62dcf3624cfb853be44b6121dc68 P 0b2b56cd5f6d4f64be7f73b6a75c439d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260810 07:15:17.035636 25033 tablet_bootstrap.cc:654] T 74aa62dcf3624cfb853be44b6121dc68 P da72d89bb77b471bbbdd19123c4776a9: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:17.035965 25033 log.cc:824] T 74aa62dcf3624cfb853be44b6121dc68 P da72d89bb77b471bbbdd19123c4776a9: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:17.036533 25031 raft_consensus.cc:348] T 74aa62dcf3624cfb853be44b6121dc68 P dc01a69a8f784435b348edba6864ce55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } } peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } }
I20260810 07:15:17.036715 25031 raft_consensus.cc:374] T 74aa62dcf3624cfb853be44b6121dc68 P dc01a69a8f784435b348edba6864ce55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:17.036834 25033 tablet_bootstrap.cc:492] T 74aa62dcf3624cfb853be44b6121dc68 P da72d89bb77b471bbbdd19123c4776a9: No bootstrap required, opened a new log
I20260810 07:15:17.036827 25031 raft_consensus.cc:729] T 74aa62dcf3624cfb853be44b6121dc68 P dc01a69a8f784435b348edba6864ce55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc01a69a8f784435b348edba6864ce55, State: Initialized, Role: FOLLOWER
I20260810 07:15:17.036906 25033 ts_tablet_manager.cc:1397] T 74aa62dcf3624cfb853be44b6121dc68 P da72d89bb77b471bbbdd19123c4776a9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:15:17.036996 25031 consensus_queue.cc:260] T 74aa62dcf3624cfb853be44b6121dc68 P dc01a69a8f784435b348edba6864ce55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } } peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } }
I20260810 07:15:17.037334 25031 ts_tablet_manager.cc:1428] T 74aa62dcf3624cfb853be44b6121dc68 P dc01a69a8f784435b348edba6864ce55: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:15:17.037572 24693 heartbeater.cc:503] Master 127.15.217.126:33813 was elected leader, sending a full tablet report...
I20260810 07:15:17.037951 25032 raft_consensus.cc:348] T 74aa62dcf3624cfb853be44b6121dc68 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } } peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } }
I20260810 07:15:17.038141 25032 raft_consensus.cc:374] T 74aa62dcf3624cfb853be44b6121dc68 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:17.038190 25032 raft_consensus.cc:729] T 74aa62dcf3624cfb853be44b6121dc68 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b2b56cd5f6d4f64be7f73b6a75c439d, State: Initialized, Role: FOLLOWER
I20260810 07:15:17.038314 25032 consensus_queue.cc:260] T 74aa62dcf3624cfb853be44b6121dc68 P 0b2b56cd5f6d4f64be7f73b6a75c439d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } } peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } }
I20260810 07:15:17.038682 24845 heartbeater.cc:503] Master 127.15.217.126:33813 was elected leader, sending a full tablet report...
I20260810 07:15:17.038936 25032 ts_tablet_manager.cc:1428] T 74aa62dcf3624cfb853be44b6121dc68 P 0b2b56cd5f6d4f64be7f73b6a75c439d: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260810 07:15:17.040431 25033 raft_consensus.cc:348] T 74aa62dcf3624cfb853be44b6121dc68 P da72d89bb77b471bbbdd19123c4776a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } } peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } }
I20260810 07:15:17.040658 25033 raft_consensus.cc:374] T 74aa62dcf3624cfb853be44b6121dc68 P da72d89bb77b471bbbdd19123c4776a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:17.040686 25033 raft_consensus.cc:729] T 74aa62dcf3624cfb853be44b6121dc68 P da72d89bb77b471bbbdd19123c4776a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da72d89bb77b471bbbdd19123c4776a9, State: Initialized, Role: FOLLOWER
I20260810 07:15:17.040844 25033 consensus_queue.cc:260] T 74aa62dcf3624cfb853be44b6121dc68 P da72d89bb77b471bbbdd19123c4776a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } } peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } }
I20260810 07:15:17.041162 25033 ts_tablet_manager.cc:1428] T 74aa62dcf3624cfb853be44b6121dc68 P da72d89bb77b471bbbdd19123c4776a9: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260810 07:15:17.041204 25002 heartbeater.cc:503] Master 127.15.217.126:33813 was elected leader, sending a full tablet report...
W20260810 07:15:17.058840 24846 tablet.cc:2369] T 74aa62dcf3624cfb853be44b6121dc68 P 0b2b56cd5f6d4f64be7f73b6a75c439d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:15:17.079375 25038 raft_consensus.cc:482] T 74aa62dcf3624cfb853be44b6121dc68 P dc01a69a8f784435b348edba6864ce55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:15:17.079563 25038 raft_consensus.cc:504] T 74aa62dcf3624cfb853be44b6121dc68 P dc01a69a8f784435b348edba6864ce55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } } peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } }
I20260810 07:15:17.080017 25038 leader_election.cc:288] T 74aa62dcf3624cfb853be44b6121dc68 P dc01a69a8f784435b348edba6864ce55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da72d89bb77b471bbbdd19123c4776a9 (127.15.217.67:40883), 0b2b56cd5f6d4f64be7f73b6a75c439d (127.15.217.66:46469)
I20260810 07:15:17.083241 25028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:17.080315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:43525 (local address 127.15.217.67:40883)
0810 07:15:17.080379 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:17.080381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:17.080679 (+ 298us) server_negotiation.cc:407] Connection header received
0810 07:15:17.080777 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:17.080779 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:17.080787 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:17.080861 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:17.081034 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.081511 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.082465 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.082849 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.082936 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:17.082946 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:17.083118 (+ 172us) server_negotiation.cc:299] Negotiation successful
0810 07:15:17.083157 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 07:15:17.083261 25029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:17.080289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:43825 (local address 127.15.217.66:46469)
0810 07:15:17.080378 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:17.080381 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:17.080393 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:15:17.080445 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:17.080446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:17.080496 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:17.080601 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:17.080830 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.081368 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.082461 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.082852 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.082949 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:17.082959 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:17.083135 (+ 176us) server_negotiation.cc:299] Negotiation successful
0810 07:15:17.083179 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 07:15:17.083290 24578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:17.080207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:46469 (local address 127.15.217.65:43825)
0810 07:15:17.080250 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:17.080264 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:17.080310 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:17.080620 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:17.080622 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:17.080625 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:17.080715 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:17.080718 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.081474 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.081475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:17.082347 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:17.082350 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.082948 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.082949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:17.083051 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:17.083054 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:15:17.083129 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 07:15:17.083179 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260810 07:15:17.083305 25041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:17.080207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:40883 (local address 127.15.217.65:43525)
0810 07:15:17.080592 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:17.080601 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:17.080705 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:17.080866 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:17.080867 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:17.080869 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:17.080959 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:17.080961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.081609 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.081610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:17.082362 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:17.082364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.082949 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.082951 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:17.083040 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:17.083043 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:15:17.083110 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 07:15:17.083154 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"mutex_wait_us":152,"thread_start_us":162,"threads_started":1}
I20260810 07:15:17.083770 24954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74aa62dcf3624cfb853be44b6121dc68" candidate_uuid: "dc01a69a8f784435b348edba6864ce55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da72d89bb77b471bbbdd19123c4776a9" is_pre_election: true
I20260810 07:15:17.083773 24794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74aa62dcf3624cfb853be44b6121dc68" candidate_uuid: "dc01a69a8f784435b348edba6864ce55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" is_pre_election: true
I20260810 07:15:17.083925 24954 raft_consensus.cc:2454] T 74aa62dcf3624cfb853be44b6121dc68 P da72d89bb77b471bbbdd19123c4776a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc01a69a8f784435b348edba6864ce55 in term 0.
I20260810 07:15:17.083966 24794 raft_consensus.cc:2454] T 74aa62dcf3624cfb853be44b6121dc68 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc01a69a8f784435b348edba6864ce55 in term 0.
I20260810 07:15:17.084299 24571 leader_election.cc:302] T 74aa62dcf3624cfb853be44b6121dc68 P dc01a69a8f784435b348edba6864ce55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: da72d89bb77b471bbbdd19123c4776a9, dc01a69a8f784435b348edba6864ce55; no voters:
I20260810 07:15:17.084443 25038 raft_consensus.cc:2781] T 74aa62dcf3624cfb853be44b6121dc68 P dc01a69a8f784435b348edba6864ce55 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:15:17.084530 25038 raft_consensus.cc:482] T 74aa62dcf3624cfb853be44b6121dc68 P dc01a69a8f784435b348edba6864ce55 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:15:17.084556 25038 raft_consensus.cc:3037] T 74aa62dcf3624cfb853be44b6121dc68 P dc01a69a8f784435b348edba6864ce55 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:17.085563 25038 raft_consensus.cc:504] T 74aa62dcf3624cfb853be44b6121dc68 P dc01a69a8f784435b348edba6864ce55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } } peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } }
I20260810 07:15:17.085736 25038 leader_election.cc:288] T 74aa62dcf3624cfb853be44b6121dc68 P dc01a69a8f784435b348edba6864ce55 [CANDIDATE]: Term 1 election: Requested vote from peers da72d89bb77b471bbbdd19123c4776a9 (127.15.217.67:40883), 0b2b56cd5f6d4f64be7f73b6a75c439d (127.15.217.66:46469)
I20260810 07:15:17.085891 24954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74aa62dcf3624cfb853be44b6121dc68" candidate_uuid: "dc01a69a8f784435b348edba6864ce55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da72d89bb77b471bbbdd19123c4776a9"
I20260810 07:15:17.085992 24954 raft_consensus.cc:3037] T 74aa62dcf3624cfb853be44b6121dc68 P da72d89bb77b471bbbdd19123c4776a9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:17.086050 24794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74aa62dcf3624cfb853be44b6121dc68" candidate_uuid: "dc01a69a8f784435b348edba6864ce55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d"
I20260810 07:15:17.086141 24794 raft_consensus.cc:3037] T 74aa62dcf3624cfb853be44b6121dc68 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:17.086814 24954 raft_consensus.cc:2454] T 74aa62dcf3624cfb853be44b6121dc68 P da72d89bb77b471bbbdd19123c4776a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc01a69a8f784435b348edba6864ce55 in term 1.
I20260810 07:15:17.086997 24571 leader_election.cc:302] T 74aa62dcf3624cfb853be44b6121dc68 P dc01a69a8f784435b348edba6864ce55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: da72d89bb77b471bbbdd19123c4776a9, dc01a69a8f784435b348edba6864ce55; no voters:
I20260810 07:15:17.087009 24794 raft_consensus.cc:2454] T 74aa62dcf3624cfb853be44b6121dc68 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc01a69a8f784435b348edba6864ce55 in term 1.
I20260810 07:15:17.087103 25038 raft_consensus.cc:2781] T 74aa62dcf3624cfb853be44b6121dc68 P dc01a69a8f784435b348edba6864ce55 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:17.087283 25038 raft_consensus.cc:686] T 74aa62dcf3624cfb853be44b6121dc68 P dc01a69a8f784435b348edba6864ce55 [term 1 LEADER]: Becoming Leader. State: Replica: dc01a69a8f784435b348edba6864ce55, State: Running, Role: LEADER
I20260810 07:15:17.087394 25038 consensus_queue.cc:237] T 74aa62dcf3624cfb853be44b6121dc68 P dc01a69a8f784435b348edba6864ce55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } } peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } }
I20260810 07:15:17.088291 24469 catalog_manager.cc:5495] T 74aa62dcf3624cfb853be44b6121dc68 P dc01a69a8f784435b348edba6864ce55 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc01a69a8f784435b348edba6864ce55 (127.15.217.65). New cstate: current_term: 1 leader_uuid: "dc01a69a8f784435b348edba6864ce55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } health_report { overall_health: UNKNOWN } } }
I20260810 07:15:17.098899 25030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:17.096683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57410 (local address 127.15.217.65:41485)
0810 07:15:17.096742 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:17.096744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:17.096751 (+ 7us) server_negotiation.cc:407] Connection header received
0810 07:15:17.096784 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:17.096785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:17.096802 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:17.096872 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:15:17.096994 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.097465 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.098194 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.098377 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.098442 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:17.098573 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:15:17.098678 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:15:17.098771 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:17.098789 (+ 18us) server_negotiation.cc:299] Negotiation successful
0810 07:15:17.098810 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 07:15:17.100734 24954 raft_consensus.cc:1261] T 74aa62dcf3624cfb853be44b6121dc68 P da72d89bb77b471bbbdd19123c4776a9 [term 1 FOLLOWER]: Refusing update from remote peer dc01a69a8f784435b348edba6864ce55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:17.100826 24794 raft_consensus.cc:1261] T 74aa62dcf3624cfb853be44b6121dc68 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 1 FOLLOWER]: Refusing update from remote peer dc01a69a8f784435b348edba6864ce55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:17.101202 25038 consensus_queue.cc:1035] T 74aa62dcf3624cfb853be44b6121dc68 P dc01a69a8f784435b348edba6864ce55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:17.101367 25042 consensus_queue.cc:1035] T 74aa62dcf3624cfb853be44b6121dc68 P dc01a69a8f784435b348edba6864ce55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:17.103742 25045 mvcc.cc:205] Tried to move back new op lower bound from 7316873695641858048 to 7316873695590608896. Current Snapshot: MvccSnapshot[applied={T|T < 7316873695641858048}]
I20260810 07:15:17.104115 25046 mvcc.cc:205] Tried to move back new op lower bound from 7316873695641858048 to 7316873695590608896. Current Snapshot: MvccSnapshot[applied={T|T < 7316873695641858048}]
I20260810 07:15:17.106525 24470 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:17.106631 24470 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:17.107425 24470 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0b336a4b1aac4e909a20ccd2a426c4a7: Sending DeleteTablet for 3 replicas of tablet 74aa62dcf3624cfb853be44b6121dc68
I20260810 07:15:17.107790 24622 tablet_service.cc:1514] Processing DeleteTablet for tablet 74aa62dcf3624cfb853be44b6121dc68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:17 UTC) from {username='oryx', principal='oryx/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:57402
I20260810 07:15:17.107825 24932 tablet_service.cc:1514] Processing DeleteTablet for tablet 74aa62dcf3624cfb853be44b6121dc68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:17 UTC) from {username='oryx', principal='oryx/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:55606
I20260810 07:15:17.108106 24774 tablet_service.cc:1514] Processing DeleteTablet for tablet 74aa62dcf3624cfb853be44b6121dc68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:17 UTC) from {username='oryx', principal='oryx/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:49630
I20260810 07:15:17.108342 25057 tablet_replica.cc:330] T 74aa62dcf3624cfb853be44b6121dc68 P 0b2b56cd5f6d4f64be7f73b6a75c439d: stopping tablet replica
I20260810 07:15:17.108330 24470 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41256:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:15:17.108486 25057 raft_consensus.cc:2229] T 74aa62dcf3624cfb853be44b6121dc68 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 1 FOLLOWER]: Raft consensus shutting down.
W20260810 07:15:17.108506 24470 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:15:17.108613 25057 raft_consensus.cc:2258] T 74aa62dcf3624cfb853be44b6121dc68 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:17.109107 25057 ts_tablet_manager.cc:1905] T 74aa62dcf3624cfb853be44b6121dc68 P 0b2b56cd5f6d4f64be7f73b6a75c439d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:17.109565 25055 tablet_replica.cc:330] T 74aa62dcf3624cfb853be44b6121dc68 P dc01a69a8f784435b348edba6864ce55: stopping tablet replica
I20260810 07:15:17.109691 25055 raft_consensus.cc:2229] T 74aa62dcf3624cfb853be44b6121dc68 P dc01a69a8f784435b348edba6864ce55 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:15:17.109795 25055 raft_consensus.cc:2258] T 74aa62dcf3624cfb853be44b6121dc68 P dc01a69a8f784435b348edba6864ce55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:17.109807 24622 tablet_service.cc:1467] Processing CreateTablet for tablet 2d7c2207b2ec4d0a97dd5a1f7c786085 (DEFAULT_TABLE table=test-table [id=56870b4da7f94b30b67bb593600d0bc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:17.109903 24774 tablet_service.cc:1467] Processing CreateTablet for tablet 2d7c2207b2ec4d0a97dd5a1f7c786085 (DEFAULT_TABLE table=test-table [id=56870b4da7f94b30b67bb593600d0bc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:17.109933 24622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d7c2207b2ec4d0a97dd5a1f7c786085. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:17.109944 24932 tablet_service.cc:1467] Processing CreateTablet for tablet 2d7c2207b2ec4d0a97dd5a1f7c786085 (DEFAULT_TABLE table=test-table [id=56870b4da7f94b30b67bb593600d0bc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:17.110016 24774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d7c2207b2ec4d0a97dd5a1f7c786085. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:17.110045 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d7c2207b2ec4d0a97dd5a1f7c786085. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:17.110936 25055 ts_tablet_manager.cc:1905] T 74aa62dcf3624cfb853be44b6121dc68 P dc01a69a8f784435b348edba6864ce55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:17.111507 25033 tablet_bootstrap.cc:492] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P da72d89bb77b471bbbdd19123c4776a9: Bootstrap starting.
I20260810 07:15:17.112195 25033 tablet_bootstrap.cc:654] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P da72d89bb77b471bbbdd19123c4776a9: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:17.112566 25031 tablet_bootstrap.cc:492] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P dc01a69a8f784435b348edba6864ce55: Bootstrap starting.
I20260810 07:15:17.112778 25056 tablet_replica.cc:330] T 74aa62dcf3624cfb853be44b6121dc68 P da72d89bb77b471bbbdd19123c4776a9: stopping tablet replica
I20260810 07:15:17.112884 25056 raft_consensus.cc:2229] T 74aa62dcf3624cfb853be44b6121dc68 P da72d89bb77b471bbbdd19123c4776a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:17.112910 25055 ts_tablet_manager.cc:1918] T 74aa62dcf3624cfb853be44b6121dc68 P dc01a69a8f784435b348edba6864ce55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:15:17.113005 25056 raft_consensus.cc:2258] T 74aa62dcf3624cfb853be44b6121dc68 P da72d89bb77b471bbbdd19123c4776a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:17.113015 25031 tablet_bootstrap.cc:654] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P dc01a69a8f784435b348edba6864ce55: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:17.113538 25055 log.cc:1196] T 74aa62dcf3624cfb853be44b6121dc68 P dc01a69a8f784435b348edba6864ce55: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-0/wal/wals/74aa62dcf3624cfb853be44b6121dc68
I20260810 07:15:17.113889 25055 ts_tablet_manager.cc:1939] T 74aa62dcf3624cfb853be44b6121dc68 P dc01a69a8f784435b348edba6864ce55: Deleting consensus metadata
I20260810 07:15:17.114383 24456 catalog_manager.cc:4841] TS dc01a69a8f784435b348edba6864ce55 (127.15.217.65:41485): tablet 74aa62dcf3624cfb853be44b6121dc68 (table test-table [id=cb2460f62af946e388706a0ab1441478]) successfully deleted
I20260810 07:15:17.114485 25032 tablet_bootstrap.cc:492] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P 0b2b56cd5f6d4f64be7f73b6a75c439d: Bootstrap starting.
I20260810 07:15:17.114751 25057 ts_tablet_manager.cc:1918] T 74aa62dcf3624cfb853be44b6121dc68 P 0b2b56cd5f6d4f64be7f73b6a75c439d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:15:17.114853 25057 log.cc:1196] T 74aa62dcf3624cfb853be44b6121dc68 P 0b2b56cd5f6d4f64be7f73b6a75c439d: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-1/wal/wals/74aa62dcf3624cfb853be44b6121dc68
I20260810 07:15:17.114931 25032 tablet_bootstrap.cc:654] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P 0b2b56cd5f6d4f64be7f73b6a75c439d: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:17.115136 25057 ts_tablet_manager.cc:1939] T 74aa62dcf3624cfb853be44b6121dc68 P 0b2b56cd5f6d4f64be7f73b6a75c439d: Deleting consensus metadata
I20260810 07:15:17.115577 25032 tablet_bootstrap.cc:492] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P 0b2b56cd5f6d4f64be7f73b6a75c439d: No bootstrap required, opened a new log
I20260810 07:15:17.115629 25032 ts_tablet_manager.cc:1397] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P 0b2b56cd5f6d4f64be7f73b6a75c439d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:17.115607 24451 catalog_manager.cc:4841] TS 0b2b56cd5f6d4f64be7f73b6a75c439d (127.15.217.66:46469): tablet 74aa62dcf3624cfb853be44b6121dc68 (table test-table [id=cb2460f62af946e388706a0ab1441478]) successfully deleted
I20260810 07:15:17.115800 25032 raft_consensus.cc:348] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } }
I20260810 07:15:17.115885 25032 raft_consensus.cc:374] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:17.115914 25032 raft_consensus.cc:729] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b2b56cd5f6d4f64be7f73b6a75c439d, State: Initialized, Role: FOLLOWER
I20260810 07:15:17.115959 25032 consensus_queue.cc:260] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P 0b2b56cd5f6d4f64be7f73b6a75c439d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } }
I20260810 07:15:17.116051 25032 ts_tablet_manager.cc:1428] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P 0b2b56cd5f6d4f64be7f73b6a75c439d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:17.116041 25056 ts_tablet_manager.cc:1905] T 74aa62dcf3624cfb853be44b6121dc68 P da72d89bb77b471bbbdd19123c4776a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:17.116976 25031 tablet_bootstrap.cc:492] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P dc01a69a8f784435b348edba6864ce55: No bootstrap required, opened a new log
I20260810 07:15:17.117044 25031 ts_tablet_manager.cc:1397] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P dc01a69a8f784435b348edba6864ce55: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260810 07:15:17.117205 25031 raft_consensus.cc:348] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P dc01a69a8f784435b348edba6864ce55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } }
I20260810 07:15:17.117282 25031 raft_consensus.cc:374] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P dc01a69a8f784435b348edba6864ce55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:17.117311 25031 raft_consensus.cc:729] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P dc01a69a8f784435b348edba6864ce55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc01a69a8f784435b348edba6864ce55, State: Initialized, Role: FOLLOWER
I20260810 07:15:17.117357 25031 consensus_queue.cc:260] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P dc01a69a8f784435b348edba6864ce55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } }
I20260810 07:15:17.117446 25031 ts_tablet_manager.cc:1428] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P dc01a69a8f784435b348edba6864ce55: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 07:15:17.117601 25056 ts_tablet_manager.cc:1918] T 74aa62dcf3624cfb853be44b6121dc68 P da72d89bb77b471bbbdd19123c4776a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:15:17.117697 25056 log.cc:1196] T 74aa62dcf3624cfb853be44b6121dc68 P da72d89bb77b471bbbdd19123c4776a9: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-2/wal/wals/74aa62dcf3624cfb853be44b6121dc68
I20260810 07:15:17.117959 25056 ts_tablet_manager.cc:1939] T 74aa62dcf3624cfb853be44b6121dc68 P da72d89bb77b471bbbdd19123c4776a9: Deleting consensus metadata
I20260810 07:15:17.118376 24452 catalog_manager.cc:4841] TS da72d89bb77b471bbbdd19123c4776a9 (127.15.217.67:40883): tablet 74aa62dcf3624cfb853be44b6121dc68 (table test-table [id=cb2460f62af946e388706a0ab1441478]) successfully deleted
I20260810 07:15:17.118594 25033 tablet_bootstrap.cc:492] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P da72d89bb77b471bbbdd19123c4776a9: No bootstrap required, opened a new log
I20260810 07:15:17.118651 25033 ts_tablet_manager.cc:1397] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P da72d89bb77b471bbbdd19123c4776a9: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20260810 07:15:17.118808 25033 raft_consensus.cc:348] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P da72d89bb77b471bbbdd19123c4776a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } }
I20260810 07:15:17.118885 25033 raft_consensus.cc:374] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P da72d89bb77b471bbbdd19123c4776a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:17.118906 25033 raft_consensus.cc:729] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P da72d89bb77b471bbbdd19123c4776a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da72d89bb77b471bbbdd19123c4776a9, State: Initialized, Role: FOLLOWER
I20260810 07:15:17.118947 25033 consensus_queue.cc:260] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P da72d89bb77b471bbbdd19123c4776a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } }
I20260810 07:15:17.119035 25033 ts_tablet_manager.cc:1428] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P da72d89bb77b471bbbdd19123c4776a9: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
W20260810 07:15:17.129117 24694 tablet.cc:2369] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P dc01a69a8f784435b348edba6864ce55: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:15:17.173523 25039 raft_consensus.cc:482] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:15:17.173632 25039 raft_consensus.cc:504] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } }
I20260810 07:15:17.174106 25039 leader_election.cc:288] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P 0b2b56cd5f6d4f64be7f73b6a75c439d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc01a69a8f784435b348edba6864ce55 (127.15.217.65:41485), da72d89bb77b471bbbdd19123c4776a9 (127.15.217.67:40883)
I20260810 07:15:17.178110 25030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:17.174419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:40405 (local address 127.15.217.65:41485)
0810 07:15:17.174470 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:17.174472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:17.174587 (+ 115us) server_negotiation.cc:407] Connection header received
0810 07:15:17.174677 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:17.174678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:17.174689 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:17.174749 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:17.174995 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.175831 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.177230 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.177638 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.177737 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:17.177748 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:17.177967 (+ 219us) server_negotiation.cc:299] Negotiation successful
0810 07:15:17.178015 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 07:15:17.178200 24728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:17.174281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:40883 (local address 127.15.217.66:56665)
0810 07:15:17.174356 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:17.174370 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:17.174441 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:17.174740 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:17.174741 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:17.174743 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:17.174824 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:17.174827 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.175905 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.175907 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:17.177159 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:17.177163 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.177901 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.177902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:17.178011 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:17.178015 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:15:17.178073 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 07:15:17.178118 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260810 07:15:17.178218 25028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:17.174528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:56665 (local address 127.15.217.67:40883)
0810 07:15:17.174589 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:17.174592 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:17.174602 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:15:17.174634 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:17.174635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:17.174647 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:17.174735 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:17.174932 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.175806 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.177272 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.177822 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.177908 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:17.177917 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:17.178094 (+ 177us) server_negotiation.cc:299] Negotiation successful
0810 07:15:17.178145 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 07:15:17.178381 25061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:17.174282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:41485 (local address 127.15.217.66:40405)
0810 07:15:17.174530 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:17.174539 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:17.174594 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:17.174807 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:17.174808 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:17.174811 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:17.174902 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:17.174904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.175934 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.175935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:17.177131 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:17.177135 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.177761 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.177763 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:17.177861 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:17.177866 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:15:17.178210 (+ 344us) client_negotiation.cc:240] Negotiation successful
0810 07:15:17.178255 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":111,"thread_start_us":66,"threads_started":1}
I20260810 07:15:17.178498 24954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d7c2207b2ec4d0a97dd5a1f7c786085" candidate_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da72d89bb77b471bbbdd19123c4776a9" is_pre_election: true
I20260810 07:15:17.178597 24954 raft_consensus.cc:2454] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P da72d89bb77b471bbbdd19123c4776a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b2b56cd5f6d4f64be7f73b6a75c439d in term 0.
I20260810 07:15:17.178704 24642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d7c2207b2ec4d0a97dd5a1f7c786085" candidate_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc01a69a8f784435b348edba6864ce55" is_pre_election: true
I20260810 07:15:17.178843 24642 raft_consensus.cc:2454] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P dc01a69a8f784435b348edba6864ce55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b2b56cd5f6d4f64be7f73b6a75c439d in term 0.
I20260810 07:15:17.178980 24721 leader_election.cc:302] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P 0b2b56cd5f6d4f64be7f73b6a75c439d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b2b56cd5f6d4f64be7f73b6a75c439d, da72d89bb77b471bbbdd19123c4776a9; no voters:
I20260810 07:15:17.179121 25039 raft_consensus.cc:2781] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:15:17.179162 25039 raft_consensus.cc:482] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:15:17.179183 25039 raft_consensus.cc:3037] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:17.180047 25039 raft_consensus.cc:504] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } }
I20260810 07:15:17.180212 25039 leader_election.cc:288] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P 0b2b56cd5f6d4f64be7f73b6a75c439d [CANDIDATE]: Term 1 election: Requested vote from peers dc01a69a8f784435b348edba6864ce55 (127.15.217.65:41485), da72d89bb77b471bbbdd19123c4776a9 (127.15.217.67:40883)
I20260810 07:15:17.180352 24642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d7c2207b2ec4d0a97dd5a1f7c786085" candidate_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc01a69a8f784435b348edba6864ce55"
I20260810 07:15:17.180429 24642 raft_consensus.cc:3037] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P dc01a69a8f784435b348edba6864ce55 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:17.180410 24954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d7c2207b2ec4d0a97dd5a1f7c786085" candidate_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da72d89bb77b471bbbdd19123c4776a9"
I20260810 07:15:17.180542 24954 raft_consensus.cc:3037] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P da72d89bb77b471bbbdd19123c4776a9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:17.181221 24954 raft_consensus.cc:2454] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P da72d89bb77b471bbbdd19123c4776a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b2b56cd5f6d4f64be7f73b6a75c439d in term 1.
I20260810 07:15:17.181435 24721 leader_election.cc:302] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P 0b2b56cd5f6d4f64be7f73b6a75c439d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b2b56cd5f6d4f64be7f73b6a75c439d, da72d89bb77b471bbbdd19123c4776a9; no voters:
I20260810 07:15:17.181450 24642 raft_consensus.cc:2454] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P dc01a69a8f784435b348edba6864ce55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b2b56cd5f6d4f64be7f73b6a75c439d in term 1.
I20260810 07:15:17.181545 25039 raft_consensus.cc:2781] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:17.181742 25039 raft_consensus.cc:686] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 1 LEADER]: Becoming Leader. State: Replica: 0b2b56cd5f6d4f64be7f73b6a75c439d, State: Running, Role: LEADER
I20260810 07:15:17.181825 25039 consensus_queue.cc:237] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P 0b2b56cd5f6d4f64be7f73b6a75c439d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } }
I20260810 07:15:17.182716 24470 catalog_manager.cc:5495] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P 0b2b56cd5f6d4f64be7f73b6a75c439d reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b2b56cd5f6d4f64be7f73b6a75c439d (127.15.217.66). New cstate: current_term: 1 leader_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } health_report { overall_health: UNKNOWN } } }
Aug 10 07:15:17 dist-test-slave-10pc krb5kdc[24420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346116, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.126@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM
I20260810 07:15:17.218138 25071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:17.209799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:33813 (local address 127.0.0.1:41264)
0810 07:15:17.209981 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:17.209993 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:17.210078 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:17.210286 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:17.210286 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:17.210701 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:17.210755 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:17.210758 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.211760 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.211761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:17.212593 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:17.212596 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.212960 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.212961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:17.213025 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:17.213575 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:17.213581 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:17.215202 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:17.216939 (+ 1737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:17.216947 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:17.216950 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:17.217159 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:17.217382 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:17.217383 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:17.217383 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:17.217449 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:17.217715 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:17.217723 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:17.217771 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 07:15:17.217849 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 07:15:17.217994 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":3,"thread_start_us":111,"threads_started":1}
I20260810 07:15:17.218138 24537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:17.209867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41264 (local address 127.15.217.126:33813)
0810 07:15:17.209927 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:17.209930 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:17.210079 (+ 149us) server_negotiation.cc:407] Connection header received
0810 07:15:17.210169 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:17.210170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:17.210183 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:17.210281 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:17.210839 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.211635 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.212683 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.212873 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.212933 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:17.215299 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:17.215300 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:17.215300 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:17.215308 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:17.216836 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:17.217251 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:17.217252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:17.217253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:17.217313 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:17.217522 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:17.217523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:17.217523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:17.217638 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:17.217711 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:17.217887 (+ 176us) server_negotiation.cc:299] Negotiation successful
0810 07:15:17.218060 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":100,"server-negotiator.queue_time_us":34}
I20260810 07:15:17.220497 24470 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:41264:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 07:15:17.220877 24470 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:15:17.222445 24622 tablet_service.cc:1467] Processing CreateTablet for tablet afd70fe115f44fc38c6585572a1e76a1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=19b910630a7a45d1a576aab12cd23e19]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 07:15:17.222450 24774 tablet_service.cc:1467] Processing CreateTablet for tablet afd70fe115f44fc38c6585572a1e76a1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=19b910630a7a45d1a576aab12cd23e19]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 07:15:17.222669 24622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afd70fe115f44fc38c6585572a1e76a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:17.222679 24774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afd70fe115f44fc38c6585572a1e76a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:17.224156 25031 tablet_bootstrap.cc:492] T afd70fe115f44fc38c6585572a1e76a1 P dc01a69a8f784435b348edba6864ce55: Bootstrap starting.
I20260810 07:15:17.224320 25032 tablet_bootstrap.cc:492] T afd70fe115f44fc38c6585572a1e76a1 P 0b2b56cd5f6d4f64be7f73b6a75c439d: Bootstrap starting.
I20260810 07:15:17.224634 25031 tablet_bootstrap.cc:654] T afd70fe115f44fc38c6585572a1e76a1 P dc01a69a8f784435b348edba6864ce55: Neither blocks nor log segments found. Creating new log.
W20260810 07:15:17.224622 25003 tablet.cc:2369] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P da72d89bb77b471bbbdd19123c4776a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:15:17.224778 25032 tablet_bootstrap.cc:654] T afd70fe115f44fc38c6585572a1e76a1 P 0b2b56cd5f6d4f64be7f73b6a75c439d: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:17.225214 24932 tablet_service.cc:1467] Processing CreateTablet for tablet afd70fe115f44fc38c6585572a1e76a1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=19b910630a7a45d1a576aab12cd23e19]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 07:15:17.225381 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afd70fe115f44fc38c6585572a1e76a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:17.225608 25031 tablet_bootstrap.cc:492] T afd70fe115f44fc38c6585572a1e76a1 P dc01a69a8f784435b348edba6864ce55: No bootstrap required, opened a new log
I20260810 07:15:17.225654 25031 ts_tablet_manager.cc:1397] T afd70fe115f44fc38c6585572a1e76a1 P dc01a69a8f784435b348edba6864ce55: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 07:15:17.225777 25031 raft_consensus.cc:348] T afd70fe115f44fc38c6585572a1e76a1 P dc01a69a8f784435b348edba6864ce55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } }
I20260810 07:15:17.225857 25031 raft_consensus.cc:374] T afd70fe115f44fc38c6585572a1e76a1 P dc01a69a8f784435b348edba6864ce55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:17.225875 25031 raft_consensus.cc:729] T afd70fe115f44fc38c6585572a1e76a1 P dc01a69a8f784435b348edba6864ce55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc01a69a8f784435b348edba6864ce55, State: Initialized, Role: FOLLOWER
I20260810 07:15:17.225919 25031 consensus_queue.cc:260] T afd70fe115f44fc38c6585572a1e76a1 P dc01a69a8f784435b348edba6864ce55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } }
I20260810 07:15:17.226001 25031 ts_tablet_manager.cc:1428] T afd70fe115f44fc38c6585572a1e76a1 P dc01a69a8f784435b348edba6864ce55: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 07:15:17.226125 25042 tablet_replica.cc:439] T afd70fe115f44fc38c6585572a1e76a1 P dc01a69a8f784435b348edba6864ce55: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } } }
I20260810 07:15:17.226212 25042 tablet_replica.cc:442] T afd70fe115f44fc38c6585572a1e76a1 P dc01a69a8f784435b348edba6864ce55: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:15:17.227093 25033 tablet_bootstrap.cc:492] T afd70fe115f44fc38c6585572a1e76a1 P da72d89bb77b471bbbdd19123c4776a9: Bootstrap starting.
I20260810 07:15:17.227535 25033 tablet_bootstrap.cc:654] T afd70fe115f44fc38c6585572a1e76a1 P da72d89bb77b471bbbdd19123c4776a9: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:17.227553 25032 tablet_bootstrap.cc:492] T afd70fe115f44fc38c6585572a1e76a1 P 0b2b56cd5f6d4f64be7f73b6a75c439d: No bootstrap required, opened a new log
I20260810 07:15:17.227627 25032 ts_tablet_manager.cc:1397] T afd70fe115f44fc38c6585572a1e76a1 P 0b2b56cd5f6d4f64be7f73b6a75c439d: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260810 07:15:17.228015 25032 raft_consensus.cc:348] T afd70fe115f44fc38c6585572a1e76a1 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } }
I20260810 07:15:17.228119 25032 raft_consensus.cc:374] T afd70fe115f44fc38c6585572a1e76a1 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:17.228149 25032 raft_consensus.cc:729] T afd70fe115f44fc38c6585572a1e76a1 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b2b56cd5f6d4f64be7f73b6a75c439d, State: Initialized, Role: FOLLOWER
I20260810 07:15:17.228194 25032 consensus_queue.cc:260] T afd70fe115f44fc38c6585572a1e76a1 P 0b2b56cd5f6d4f64be7f73b6a75c439d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } }
I20260810 07:15:17.228238 25033 tablet_bootstrap.cc:492] T afd70fe115f44fc38c6585572a1e76a1 P da72d89bb77b471bbbdd19123c4776a9: No bootstrap required, opened a new log
I20260810 07:15:17.228289 25033 ts_tablet_manager.cc:1397] T afd70fe115f44fc38c6585572a1e76a1 P da72d89bb77b471bbbdd19123c4776a9: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 07:15:17.228293 25032 ts_tablet_manager.cc:1428] T afd70fe115f44fc38c6585572a1e76a1 P 0b2b56cd5f6d4f64be7f73b6a75c439d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260810 07:15:17.228428 25033 raft_consensus.cc:348] T afd70fe115f44fc38c6585572a1e76a1 P da72d89bb77b471bbbdd19123c4776a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } }
I20260810 07:15:17.228526 25033 raft_consensus.cc:374] T afd70fe115f44fc38c6585572a1e76a1 P da72d89bb77b471bbbdd19123c4776a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:17.228441 25039 tablet_replica.cc:439] T afd70fe115f44fc38c6585572a1e76a1 P 0b2b56cd5f6d4f64be7f73b6a75c439d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } } }
I20260810 07:15:17.228546 25033 raft_consensus.cc:729] T afd70fe115f44fc38c6585572a1e76a1 P da72d89bb77b471bbbdd19123c4776a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da72d89bb77b471bbbdd19123c4776a9, State: Initialized, Role: FOLLOWER
I20260810 07:15:17.228555 25039 tablet_replica.cc:442] T afd70fe115f44fc38c6585572a1e76a1 P 0b2b56cd5f6d4f64be7f73b6a75c439d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:15:17.228590 25033 consensus_queue.cc:260] T afd70fe115f44fc38c6585572a1e76a1 P da72d89bb77b471bbbdd19123c4776a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } }
I20260810 07:15:17.228675 25033 ts_tablet_manager.cc:1428] T afd70fe115f44fc38c6585572a1e76a1 P da72d89bb77b471bbbdd19123c4776a9: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 07:15:17.228811 25040 tablet_replica.cc:439] T afd70fe115f44fc38c6585572a1e76a1 P da72d89bb77b471bbbdd19123c4776a9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } } }
I20260810 07:15:17.228880 25040 tablet_replica.cc:442] T afd70fe115f44fc38c6585572a1e76a1 P da72d89bb77b471bbbdd19123c4776a9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:15:17.312568 25039 raft_consensus.cc:482] T afd70fe115f44fc38c6585572a1e76a1 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:15:17.312706 25039 raft_consensus.cc:504] T afd70fe115f44fc38c6585572a1e76a1 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } }
I20260810 07:15:17.312932 25039 leader_election.cc:288] T afd70fe115f44fc38c6585572a1e76a1 P 0b2b56cd5f6d4f64be7f73b6a75c439d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc01a69a8f784435b348edba6864ce55 (127.15.217.65:41485), da72d89bb77b471bbbdd19123c4776a9 (127.15.217.67:40883)
I20260810 07:15:17.313211 24954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afd70fe115f44fc38c6585572a1e76a1" candidate_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da72d89bb77b471bbbdd19123c4776a9" is_pre_election: true
I20260810 07:15:17.313211 24642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afd70fe115f44fc38c6585572a1e76a1" candidate_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc01a69a8f784435b348edba6864ce55" is_pre_election: true
I20260810 07:15:17.313336 24954 raft_consensus.cc:2454] T afd70fe115f44fc38c6585572a1e76a1 P da72d89bb77b471bbbdd19123c4776a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b2b56cd5f6d4f64be7f73b6a75c439d in term 0.
I20260810 07:15:17.313336 24642 raft_consensus.cc:2454] T afd70fe115f44fc38c6585572a1e76a1 P dc01a69a8f784435b348edba6864ce55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b2b56cd5f6d4f64be7f73b6a75c439d in term 0.
I20260810 07:15:17.313534 24723 leader_election.cc:302] T afd70fe115f44fc38c6585572a1e76a1 P 0b2b56cd5f6d4f64be7f73b6a75c439d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b2b56cd5f6d4f64be7f73b6a75c439d, dc01a69a8f784435b348edba6864ce55; no voters:
I20260810 07:15:17.313663 25039 raft_consensus.cc:2781] T afd70fe115f44fc38c6585572a1e76a1 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:15:17.313717 25039 raft_consensus.cc:482] T afd70fe115f44fc38c6585572a1e76a1 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:15:17.313750 25039 raft_consensus.cc:3037] T afd70fe115f44fc38c6585572a1e76a1 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:17.314484 25039 raft_consensus.cc:504] T afd70fe115f44fc38c6585572a1e76a1 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } }
I20260810 07:15:17.314637 25039 leader_election.cc:288] T afd70fe115f44fc38c6585572a1e76a1 P 0b2b56cd5f6d4f64be7f73b6a75c439d [CANDIDATE]: Term 1 election: Requested vote from peers dc01a69a8f784435b348edba6864ce55 (127.15.217.65:41485), da72d89bb77b471bbbdd19123c4776a9 (127.15.217.67:40883)
I20260810 07:15:17.314841 24642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afd70fe115f44fc38c6585572a1e76a1" candidate_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc01a69a8f784435b348edba6864ce55"
I20260810 07:15:17.314853 24954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afd70fe115f44fc38c6585572a1e76a1" candidate_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da72d89bb77b471bbbdd19123c4776a9"
I20260810 07:15:17.314934 24954 raft_consensus.cc:3037] T afd70fe115f44fc38c6585572a1e76a1 P da72d89bb77b471bbbdd19123c4776a9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:17.314936 24642 raft_consensus.cc:3037] T afd70fe115f44fc38c6585572a1e76a1 P dc01a69a8f784435b348edba6864ce55 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:17.315560 24954 raft_consensus.cc:2454] T afd70fe115f44fc38c6585572a1e76a1 P da72d89bb77b471bbbdd19123c4776a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b2b56cd5f6d4f64be7f73b6a75c439d in term 1.
I20260810 07:15:17.315738 24642 raft_consensus.cc:2454] T afd70fe115f44fc38c6585572a1e76a1 P dc01a69a8f784435b348edba6864ce55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b2b56cd5f6d4f64be7f73b6a75c439d in term 1.
I20260810 07:15:17.315750 24721 leader_election.cc:302] T afd70fe115f44fc38c6585572a1e76a1 P 0b2b56cd5f6d4f64be7f73b6a75c439d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b2b56cd5f6d4f64be7f73b6a75c439d, da72d89bb77b471bbbdd19123c4776a9; no voters:
I20260810 07:15:17.315858 25039 raft_consensus.cc:2781] T afd70fe115f44fc38c6585572a1e76a1 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:17.315912 25039 raft_consensus.cc:686] T afd70fe115f44fc38c6585572a1e76a1 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 1 LEADER]: Becoming Leader. State: Replica: 0b2b56cd5f6d4f64be7f73b6a75c439d, State: Running, Role: LEADER
I20260810 07:15:17.315964 25039 consensus_queue.cc:237] T afd70fe115f44fc38c6585572a1e76a1 P 0b2b56cd5f6d4f64be7f73b6a75c439d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } }
I20260810 07:15:17.316294 25063 tablet_replica.cc:439] T afd70fe115f44fc38c6585572a1e76a1 P 0b2b56cd5f6d4f64be7f73b6a75c439d: TxnStatusTablet state changed. Reason: New leader 0b2b56cd5f6d4f64be7f73b6a75c439d. Latest consensus state: current_term: 1 leader_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } } }
I20260810 07:15:17.316398 25063 tablet_replica.cc:442] T afd70fe115f44fc38c6585572a1e76a1 P 0b2b56cd5f6d4f64be7f73b6a75c439d: This TxnStatusTablet replica's current role is: LEADER
I20260810 07:15:17.316675 25077 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 07:15:17.316697 24470 catalog_manager.cc:5495] T afd70fe115f44fc38c6585572a1e76a1 P 0b2b56cd5f6d4f64be7f73b6a75c439d reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b2b56cd5f6d4f64be7f73b6a75c439d (127.15.217.66). New cstate: current_term: 1 leader_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } health_report { overall_health: UNKNOWN } } }
I20260810 07:15:17.316819 25077 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 07:15:17.323933 25071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:17.320841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:46469 (local address 127.0.0.1:49636)
0810 07:15:17.320920 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:17.320933 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:17.320987 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:17.321195 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:17.321196 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:17.321197 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:15:17.321244 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:17.321246 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.322159 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.322159 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:17.322809 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:17.322812 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.323318 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.323319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:17.323372 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:17.323425 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:15:17.323789 (+ 364us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:15:17.323796 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 07:15:17.323824 (+ 28us) client_negotiation.cc:240] Negotiation successful
0810 07:15:17.323869 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260810 07:15:17.323939 25029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:17.320911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49636 (local address 127.15.217.66:46469)
0810 07:15:17.320953 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:17.320956 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:17.320966 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:15:17.321076 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:17.321077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:17.321098 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:17.321178 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:15:17.321313 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.322076 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.322895 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.323239 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.323316 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:17.323539 (+ 223us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:15:17.323695 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:15:17.323806 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:17.323838 (+ 32us) server_negotiation.cc:299] Negotiation successful
0810 07:15:17.323862 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 07:15:17.325248 24954 raft_consensus.cc:1261] T afd70fe115f44fc38c6585572a1e76a1 P da72d89bb77b471bbbdd19123c4776a9 [term 1 FOLLOWER]: Refusing update from remote peer 0b2b56cd5f6d4f64be7f73b6a75c439d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:17.325421 24642 raft_consensus.cc:1261] T afd70fe115f44fc38c6585572a1e76a1 P dc01a69a8f784435b348edba6864ce55 [term 1 FOLLOWER]: Refusing update from remote peer 0b2b56cd5f6d4f64be7f73b6a75c439d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:17.325533 25063 consensus_queue.cc:1035] T afd70fe115f44fc38c6585572a1e76a1 P 0b2b56cd5f6d4f64be7f73b6a75c439d [LEADER]: Connected to new peer: Peer: permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:17.325717 25063 consensus_queue.cc:1035] T afd70fe115f44fc38c6585572a1e76a1 P 0b2b56cd5f6d4f64be7f73b6a75c439d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:17.326335 25040 tablet_replica.cc:439] T afd70fe115f44fc38c6585572a1e76a1 P da72d89bb77b471bbbdd19123c4776a9: TxnStatusTablet state changed. Reason: New leader 0b2b56cd5f6d4f64be7f73b6a75c439d. Latest consensus state: current_term: 1 leader_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } } }
I20260810 07:15:17.326439 25040 tablet_replica.cc:442] T afd70fe115f44fc38c6585572a1e76a1 P da72d89bb77b471bbbdd19123c4776a9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:15:17.326453 25042 tablet_replica.cc:439] T afd70fe115f44fc38c6585572a1e76a1 P dc01a69a8f784435b348edba6864ce55: TxnStatusTablet state changed. Reason: New leader 0b2b56cd5f6d4f64be7f73b6a75c439d. Latest consensus state: current_term: 1 leader_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } } }
I20260810 07:15:17.326539 25042 tablet_replica.cc:442] T afd70fe115f44fc38c6585572a1e76a1 P dc01a69a8f784435b348edba6864ce55: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:15:17.327221 25039 tablet_replica.cc:439] T afd70fe115f44fc38c6585572a1e76a1 P 0b2b56cd5f6d4f64be7f73b6a75c439d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } } }
I20260810 07:15:17.327306 25039 tablet_replica.cc:442] T afd70fe115f44fc38c6585572a1e76a1 P 0b2b56cd5f6d4f64be7f73b6a75c439d: This TxnStatusTablet replica's current role is: LEADER
I20260810 07:15:17.327540 25042 tablet_replica.cc:439] T afd70fe115f44fc38c6585572a1e76a1 P dc01a69a8f784435b348edba6864ce55: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } } }
I20260810 07:15:17.327637 25042 tablet_replica.cc:442] T afd70fe115f44fc38c6585572a1e76a1 P dc01a69a8f784435b348edba6864ce55: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:15:17.331789 25029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:17.329322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49642 (local address 127.15.217.66:46469)
0810 07:15:17.329368 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:17.329370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:17.329379 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:15:17.329407 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:17.329409 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:17.329422 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:17.329513 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:15:17.329602 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.330321 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.331014 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.331341 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.331407 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:17.331540 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:15:17.331576 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:15:17.331628 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:17.331697 (+ 69us) server_negotiation.cc:299] Negotiation successful
0810 07:15:17.331719 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 07:15:17.337179 25039 tablet_replica.cc:439] T afd70fe115f44fc38c6585572a1e76a1 P 0b2b56cd5f6d4f64be7f73b6a75c439d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } } }
I20260810 07:15:17.337283 25029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:17.334483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:53343 (local address 127.15.217.66:46469)
0810 07:15:17.334541 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:17.334543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:17.334666 (+ 123us) server_negotiation.cc:407] Connection header received
0810 07:15:17.334697 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:17.334698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:17.334708 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:17.334773 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:15:17.334910 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.335625 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.336434 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.336737 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.336814 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:17.337024 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:15:17.337060 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:15:17.337114 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:17.337192 (+ 78us) server_negotiation.cc:299] Negotiation successful
0810 07:15:17.337213 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 07:15:17.337412 25047 mvcc.cc:205] Tried to move back new op lower bound from 7316873696561979392 to 7316873696526684160. Current Snapshot: MvccSnapshot[applied={T|T < 7316873696561979392}]
I20260810 07:15:17.337527 25084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:17.334398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:46469 (local address 127.15.217.66:53343)
0810 07:15:17.334587 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:17.334598 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:17.334659 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:17.334775 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:17.334775 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:17.334777 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:15:17.334845 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:17.334846 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.335727 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.335729 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:17.336349 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:17.336351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.336832 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.336834 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:17.336911 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:17.336930 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:15:17.337115 (+ 185us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:15:17.337117 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 07:15:17.337393 (+ 276us) client_negotiation.cc:240] Negotiation successful
0810 07:15:17.337435 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":121,"threads_started":1}
I20260810 07:15:17.337489 25040 tablet_replica.cc:439] T afd70fe115f44fc38c6585572a1e76a1 P da72d89bb77b471bbbdd19123c4776a9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } } }
I20260810 07:15:17.337574 25040 tablet_replica.cc:442] T afd70fe115f44fc38c6585572a1e76a1 P da72d89bb77b471bbbdd19123c4776a9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:15:17.337286 25039 tablet_replica.cc:442] T afd70fe115f44fc38c6585572a1e76a1 P 0b2b56cd5f6d4f64be7f73b6a75c439d: This TxnStatusTablet replica's current role is: LEADER
I20260810 07:15:17.339866 24954 raft_consensus.cc:1261] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P da72d89bb77b471bbbdd19123c4776a9 [term 1 FOLLOWER]: Refusing update from remote peer 0b2b56cd5f6d4f64be7f73b6a75c439d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:17.339903 24642 raft_consensus.cc:1261] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P dc01a69a8f784435b348edba6864ce55 [term 1 FOLLOWER]: Refusing update from remote peer 0b2b56cd5f6d4f64be7f73b6a75c439d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:17.340099 25085 consensus_queue.cc:1035] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P 0b2b56cd5f6d4f64be7f73b6a75c439d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:17.340245 25063 consensus_queue.cc:1035] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P 0b2b56cd5f6d4f64be7f73b6a75c439d [LEADER]: Connected to new peer: Peer: permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:17.354792 24470 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:17.354913 24470 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:17.355588 24470 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0b336a4b1aac4e909a20ccd2a426c4a7: Sending DeleteTablet for 3 replicas of tablet 2d7c2207b2ec4d0a97dd5a1f7c786085
I20260810 07:15:17.355897 24622 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d7c2207b2ec4d0a97dd5a1f7c786085 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:17 UTC) from {username='oryx', principal='oryx/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:57402
I20260810 07:15:17.356020 25055 tablet_replica.cc:330] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P dc01a69a8f784435b348edba6864ce55: stopping tablet replica
I20260810 07:15:17.356097 25055 raft_consensus.cc:2229] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P dc01a69a8f784435b348edba6864ce55 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:17.356079 24932 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d7c2207b2ec4d0a97dd5a1f7c786085 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:17 UTC) from {username='oryx', principal='oryx/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:55606
I20260810 07:15:17.356144 25055 raft_consensus.cc:2258] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P dc01a69a8f784435b348edba6864ce55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:17.356169 25056 tablet_replica.cc:330] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P da72d89bb77b471bbbdd19123c4776a9: stopping tablet replica
I20260810 07:15:17.356216 25056 raft_consensus.cc:2229] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P da72d89bb77b471bbbdd19123c4776a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:17.356263 25056 raft_consensus.cc:2258] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P da72d89bb77b471bbbdd19123c4776a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:17.356527 25055 ts_tablet_manager.cc:1905] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P dc01a69a8f784435b348edba6864ce55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:17.356634 24773 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d7c2207b2ec4d0a97dd5a1f7c786085 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:17 UTC) from {username='oryx', principal='oryx/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:49630
I20260810 07:15:17.356719 25057 tablet_replica.cc:330] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P 0b2b56cd5f6d4f64be7f73b6a75c439d: stopping tablet replica
I20260810 07:15:17.356765 25057 raft_consensus.cc:2229] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:15:17.356853 25057 raft_consensus.cc:2258] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:17.357347 25056 ts_tablet_manager.cc:1905] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P da72d89bb77b471bbbdd19123c4776a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:17.358132 25057 ts_tablet_manager.cc:1905] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P 0b2b56cd5f6d4f64be7f73b6a75c439d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:17.358474 25055 ts_tablet_manager.cc:1918] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P dc01a69a8f784435b348edba6864ce55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:15:17.358526 25055 log.cc:1196] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P dc01a69a8f784435b348edba6864ce55: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-0/wal/wals/2d7c2207b2ec4d0a97dd5a1f7c786085
I20260810 07:15:17.358744 25055 ts_tablet_manager.cc:1939] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P dc01a69a8f784435b348edba6864ce55: Deleting consensus metadata
I20260810 07:15:17.359117 25056 ts_tablet_manager.cc:1918] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P da72d89bb77b471bbbdd19123c4776a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:15:17.359174 25056 log.cc:1196] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P da72d89bb77b471bbbdd19123c4776a9: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-2/wal/wals/2d7c2207b2ec4d0a97dd5a1f7c786085
I20260810 07:15:17.359212 24456 catalog_manager.cc:4841] TS dc01a69a8f784435b348edba6864ce55 (127.15.217.65:41485): tablet 2d7c2207b2ec4d0a97dd5a1f7c786085 (table test-table [id=56870b4da7f94b30b67bb593600d0bc7]) successfully deleted
I20260810 07:15:17.359387 25056 ts_tablet_manager.cc:1939] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P da72d89bb77b471bbbdd19123c4776a9: Deleting consensus metadata
I20260810 07:15:17.359725 25057 ts_tablet_manager.cc:1918] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P 0b2b56cd5f6d4f64be7f73b6a75c439d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:15:17.359782 25057 log.cc:1196] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P 0b2b56cd5f6d4f64be7f73b6a75c439d: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-1/wal/wals/2d7c2207b2ec4d0a97dd5a1f7c786085
I20260810 07:15:17.359841 24452 catalog_manager.cc:4841] TS da72d89bb77b471bbbdd19123c4776a9 (127.15.217.67:40883): tablet 2d7c2207b2ec4d0a97dd5a1f7c786085 (table test-table [id=56870b4da7f94b30b67bb593600d0bc7]) successfully deleted
I20260810 07:15:17.360008 25057 ts_tablet_manager.cc:1939] T 2d7c2207b2ec4d0a97dd5a1f7c786085 P 0b2b56cd5f6d4f64be7f73b6a75c439d: Deleting consensus metadata
I20260810 07:15:17.360342 24451 catalog_manager.cc:4841] TS 0b2b56cd5f6d4f64be7f73b6a75c439d (127.15.217.66:46469): tablet 2d7c2207b2ec4d0a97dd5a1f7c786085 (table test-table [id=56870b4da7f94b30b67bb593600d0bc7]) successfully deleted
Aug 10 07:15:17 dist-test-slave-10pc krb5kdc[24420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 07:15:17 dist-test-slave-10pc krb5kdc[24420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:17 dist-test-slave-10pc krb5kdc[24420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
W20260810 07:15:17.383988 24537 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:17.377382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41276 (local address 127.15.217.126:33813)
0810 07:15:17.377465 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:17.377468 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:17.377615 (+ 147us) server_negotiation.cc:407] Connection header received
0810 07:15:17.377705 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:17.377706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:17.377717 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:17.377771 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:17.378526 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.379147 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.380300 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.380538 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.380591 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:17.383915 (+ 3324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41276: BlockingRecv error: recv got EOF from 127.0.0.1:41276 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 10 07:15:17 dist-test-slave-10pc krb5kdc[24420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM
I20260810 07:15:17.392884 24537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:17.385170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41292 (local address 127.15.217.126:33813)
0810 07:15:17.385209 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:17.385211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:17.385282 (+ 71us) server_negotiation.cc:407] Connection header received
0810 07:15:17.385352 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:17.385353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:17.385364 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:17.385417 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:17.385886 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.386654 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.387327 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.387644 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.387718 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:17.390072 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:17.390074 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:17.390074 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:17.390082 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:17.391664 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:17.391998 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:17.391999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:17.391999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:17.392063 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:17.392260 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:17.392260 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:17.392261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:17.392398 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:17.392543 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:17.392696 (+ 153us) server_negotiation.cc:299] Negotiation successful
0810 07:15:17.392779 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 07:15:17.394536 24470 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41292:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:15:17.394726 24470 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:15:17.396222 24622 tablet_service.cc:1467] Processing CreateTablet for tablet 0327cbdf92324218b5e63f97580cf985 (DEFAULT_TABLE table=test-table [id=4dbad75d0d494faaa57d8b7a39daa3cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:17.396382 24622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0327cbdf92324218b5e63f97580cf985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:17.396566 24773 tablet_service.cc:1467] Processing CreateTablet for tablet 0327cbdf92324218b5e63f97580cf985 (DEFAULT_TABLE table=test-table [id=4dbad75d0d494faaa57d8b7a39daa3cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:17.396624 24932 tablet_service.cc:1467] Processing CreateTablet for tablet 0327cbdf92324218b5e63f97580cf985 (DEFAULT_TABLE table=test-table [id=4dbad75d0d494faaa57d8b7a39daa3cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:17.396704 24773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0327cbdf92324218b5e63f97580cf985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:17.396792 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0327cbdf92324218b5e63f97580cf985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:17.398007 25033 tablet_bootstrap.cc:492] T 0327cbdf92324218b5e63f97580cf985 P da72d89bb77b471bbbdd19123c4776a9: Bootstrap starting.
I20260810 07:15:17.398140 25032 tablet_bootstrap.cc:492] T 0327cbdf92324218b5e63f97580cf985 P 0b2b56cd5f6d4f64be7f73b6a75c439d: Bootstrap starting.
I20260810 07:15:17.398470 25033 tablet_bootstrap.cc:654] T 0327cbdf92324218b5e63f97580cf985 P da72d89bb77b471bbbdd19123c4776a9: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:17.398550 25032 tablet_bootstrap.cc:654] T 0327cbdf92324218b5e63f97580cf985 P 0b2b56cd5f6d4f64be7f73b6a75c439d: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:17.399268 25032 tablet_bootstrap.cc:492] T 0327cbdf92324218b5e63f97580cf985 P 0b2b56cd5f6d4f64be7f73b6a75c439d: No bootstrap required, opened a new log
I20260810 07:15:17.399327 25032 ts_tablet_manager.cc:1397] T 0327cbdf92324218b5e63f97580cf985 P 0b2b56cd5f6d4f64be7f73b6a75c439d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 07:15:17.399463 25032 raft_consensus.cc:348] T 0327cbdf92324218b5e63f97580cf985 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } }
I20260810 07:15:17.399539 25032 raft_consensus.cc:374] T 0327cbdf92324218b5e63f97580cf985 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:17.399554 25032 raft_consensus.cc:729] T 0327cbdf92324218b5e63f97580cf985 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b2b56cd5f6d4f64be7f73b6a75c439d, State: Initialized, Role: FOLLOWER
I20260810 07:15:17.399586 25032 consensus_queue.cc:260] T 0327cbdf92324218b5e63f97580cf985 P 0b2b56cd5f6d4f64be7f73b6a75c439d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } }
I20260810 07:15:17.399680 25032 ts_tablet_manager.cc:1428] T 0327cbdf92324218b5e63f97580cf985 P 0b2b56cd5f6d4f64be7f73b6a75c439d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:17.400827 25031 tablet_bootstrap.cc:492] T 0327cbdf92324218b5e63f97580cf985 P dc01a69a8f784435b348edba6864ce55: Bootstrap starting.
I20260810 07:15:17.400908 25033 tablet_bootstrap.cc:492] T 0327cbdf92324218b5e63f97580cf985 P da72d89bb77b471bbbdd19123c4776a9: No bootstrap required, opened a new log
I20260810 07:15:17.400957 25033 ts_tablet_manager.cc:1397] T 0327cbdf92324218b5e63f97580cf985 P da72d89bb77b471bbbdd19123c4776a9: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260810 07:15:17.401100 25033 raft_consensus.cc:348] T 0327cbdf92324218b5e63f97580cf985 P da72d89bb77b471bbbdd19123c4776a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } }
I20260810 07:15:17.401149 25033 raft_consensus.cc:374] T 0327cbdf92324218b5e63f97580cf985 P da72d89bb77b471bbbdd19123c4776a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:17.401132 25031 tablet_bootstrap.cc:654] T 0327cbdf92324218b5e63f97580cf985 P dc01a69a8f784435b348edba6864ce55: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:17.401161 25033 raft_consensus.cc:729] T 0327cbdf92324218b5e63f97580cf985 P da72d89bb77b471bbbdd19123c4776a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da72d89bb77b471bbbdd19123c4776a9, State: Initialized, Role: FOLLOWER
I20260810 07:15:17.401228 25033 consensus_queue.cc:260] T 0327cbdf92324218b5e63f97580cf985 P da72d89bb77b471bbbdd19123c4776a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } }
I20260810 07:15:17.401355 25033 ts_tablet_manager.cc:1428] T 0327cbdf92324218b5e63f97580cf985 P da72d89bb77b471bbbdd19123c4776a9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:17.402122 25031 tablet_bootstrap.cc:492] T 0327cbdf92324218b5e63f97580cf985 P dc01a69a8f784435b348edba6864ce55: No bootstrap required, opened a new log
I20260810 07:15:17.402175 25031 ts_tablet_manager.cc:1397] T 0327cbdf92324218b5e63f97580cf985 P dc01a69a8f784435b348edba6864ce55: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 07:15:17.402283 25031 raft_consensus.cc:348] T 0327cbdf92324218b5e63f97580cf985 P dc01a69a8f784435b348edba6864ce55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } }
I20260810 07:15:17.402351 25031 raft_consensus.cc:374] T 0327cbdf92324218b5e63f97580cf985 P dc01a69a8f784435b348edba6864ce55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:17.402369 25031 raft_consensus.cc:729] T 0327cbdf92324218b5e63f97580cf985 P dc01a69a8f784435b348edba6864ce55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc01a69a8f784435b348edba6864ce55, State: Initialized, Role: FOLLOWER
I20260810 07:15:17.402408 25031 consensus_queue.cc:260] T 0327cbdf92324218b5e63f97580cf985 P dc01a69a8f784435b348edba6864ce55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } }
I20260810 07:15:17.402490 25031 ts_tablet_manager.cc:1428] T 0327cbdf92324218b5e63f97580cf985 P dc01a69a8f784435b348edba6864ce55: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:17.449862 25042 raft_consensus.cc:482] T 0327cbdf92324218b5e63f97580cf985 P dc01a69a8f784435b348edba6864ce55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:15:17.450017 25042 raft_consensus.cc:504] T 0327cbdf92324218b5e63f97580cf985 P dc01a69a8f784435b348edba6864ce55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } }
I20260810 07:15:17.450253 25042 leader_election.cc:288] T 0327cbdf92324218b5e63f97580cf985 P dc01a69a8f784435b348edba6864ce55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b2b56cd5f6d4f64be7f73b6a75c439d (127.15.217.66:46469), da72d89bb77b471bbbdd19123c4776a9 (127.15.217.67:40883)
I20260810 07:15:17.450495 24794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0327cbdf92324218b5e63f97580cf985" candidate_uuid: "dc01a69a8f784435b348edba6864ce55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" is_pre_election: true
I20260810 07:15:17.450517 24954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0327cbdf92324218b5e63f97580cf985" candidate_uuid: "dc01a69a8f784435b348edba6864ce55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da72d89bb77b471bbbdd19123c4776a9" is_pre_election: true
I20260810 07:15:17.450604 24794 raft_consensus.cc:2454] T 0327cbdf92324218b5e63f97580cf985 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc01a69a8f784435b348edba6864ce55 in term 0.
I20260810 07:15:17.450605 24954 raft_consensus.cc:2454] T 0327cbdf92324218b5e63f97580cf985 P da72d89bb77b471bbbdd19123c4776a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc01a69a8f784435b348edba6864ce55 in term 0.
I20260810 07:15:17.450796 24570 leader_election.cc:302] T 0327cbdf92324218b5e63f97580cf985 P dc01a69a8f784435b348edba6864ce55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b2b56cd5f6d4f64be7f73b6a75c439d, dc01a69a8f784435b348edba6864ce55; no voters:
I20260810 07:15:17.450901 25042 raft_consensus.cc:2781] T 0327cbdf92324218b5e63f97580cf985 P dc01a69a8f784435b348edba6864ce55 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:15:17.450940 25042 raft_consensus.cc:482] T 0327cbdf92324218b5e63f97580cf985 P dc01a69a8f784435b348edba6864ce55 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:15:17.450971 25042 raft_consensus.cc:3037] T 0327cbdf92324218b5e63f97580cf985 P dc01a69a8f784435b348edba6864ce55 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:17.451843 25042 raft_consensus.cc:504] T 0327cbdf92324218b5e63f97580cf985 P dc01a69a8f784435b348edba6864ce55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } }
I20260810 07:15:17.451985 25042 leader_election.cc:288] T 0327cbdf92324218b5e63f97580cf985 P dc01a69a8f784435b348edba6864ce55 [CANDIDATE]: Term 1 election: Requested vote from peers 0b2b56cd5f6d4f64be7f73b6a75c439d (127.15.217.66:46469), da72d89bb77b471bbbdd19123c4776a9 (127.15.217.67:40883)
I20260810 07:15:17.452186 24954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0327cbdf92324218b5e63f97580cf985" candidate_uuid: "dc01a69a8f784435b348edba6864ce55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da72d89bb77b471bbbdd19123c4776a9"
I20260810 07:15:17.452180 24794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0327cbdf92324218b5e63f97580cf985" candidate_uuid: "dc01a69a8f784435b348edba6864ce55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d"
I20260810 07:15:17.452261 24954 raft_consensus.cc:3037] T 0327cbdf92324218b5e63f97580cf985 P da72d89bb77b471bbbdd19123c4776a9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:17.452292 24794 raft_consensus.cc:3037] T 0327cbdf92324218b5e63f97580cf985 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:17.453003 24954 raft_consensus.cc:2454] T 0327cbdf92324218b5e63f97580cf985 P da72d89bb77b471bbbdd19123c4776a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc01a69a8f784435b348edba6864ce55 in term 1.
I20260810 07:15:17.453189 24571 leader_election.cc:302] T 0327cbdf92324218b5e63f97580cf985 P dc01a69a8f784435b348edba6864ce55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: da72d89bb77b471bbbdd19123c4776a9, dc01a69a8f784435b348edba6864ce55; no voters:
I20260810 07:15:17.453210 24794 raft_consensus.cc:2454] T 0327cbdf92324218b5e63f97580cf985 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc01a69a8f784435b348edba6864ce55 in term 1.
I20260810 07:15:17.453301 25042 raft_consensus.cc:2781] T 0327cbdf92324218b5e63f97580cf985 P dc01a69a8f784435b348edba6864ce55 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:17.453349 25042 raft_consensus.cc:686] T 0327cbdf92324218b5e63f97580cf985 P dc01a69a8f784435b348edba6864ce55 [term 1 LEADER]: Becoming Leader. State: Replica: dc01a69a8f784435b348edba6864ce55, State: Running, Role: LEADER
I20260810 07:15:17.453395 25042 consensus_queue.cc:237] T 0327cbdf92324218b5e63f97580cf985 P dc01a69a8f784435b348edba6864ce55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } }
I20260810 07:15:17.454108 24469 catalog_manager.cc:5495] T 0327cbdf92324218b5e63f97580cf985 P dc01a69a8f784435b348edba6864ce55 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc01a69a8f784435b348edba6864ce55 (127.15.217.65). New cstate: current_term: 1 leader_uuid: "dc01a69a8f784435b348edba6864ce55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } health_report { overall_health: UNKNOWN } } }
I20260810 07:15:17.473668 25030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:17.471255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57422 (local address 127.15.217.65:41485)
0810 07:15:17.471311 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:17.471314 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:17.471325 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:15:17.471379 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:17.471380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:17.471391 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:17.471456 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:15:17.471601 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.472084 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.472955 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.473153 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.473243 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:17.473385 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:15:17.473423 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:15:17.473483 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:17.473545 (+ 62us) server_negotiation.cc:299] Negotiation successful
0810 07:15:17.473571 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 07:15:17.474565 24954 raft_consensus.cc:1261] T 0327cbdf92324218b5e63f97580cf985 P da72d89bb77b471bbbdd19123c4776a9 [term 1 FOLLOWER]: Refusing update from remote peer dc01a69a8f784435b348edba6864ce55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:17.474630 24794 raft_consensus.cc:1261] T 0327cbdf92324218b5e63f97580cf985 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 1 FOLLOWER]: Refusing update from remote peer dc01a69a8f784435b348edba6864ce55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:17.474869 25038 consensus_queue.cc:1035] T 0327cbdf92324218b5e63f97580cf985 P dc01a69a8f784435b348edba6864ce55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:17.475172 25038 consensus_queue.cc:1035] T 0327cbdf92324218b5e63f97580cf985 P dc01a69a8f784435b348edba6864ce55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:17.478377 24469 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41292:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:17.478482 24469 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41292:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:17.479131 24469 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0b336a4b1aac4e909a20ccd2a426c4a7: Sending DeleteTablet for 3 replicas of tablet 0327cbdf92324218b5e63f97580cf985
I20260810 07:15:17.479406 24932 tablet_service.cc:1514] Processing DeleteTablet for tablet 0327cbdf92324218b5e63f97580cf985 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:17 UTC) from {username='oryx', principal='oryx/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:55606
I20260810 07:15:17.479424 24622 tablet_service.cc:1514] Processing DeleteTablet for tablet 0327cbdf92324218b5e63f97580cf985 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:17 UTC) from {username='oryx', principal='oryx/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:57402
I20260810 07:15:17.479501 25056 tablet_replica.cc:330] T 0327cbdf92324218b5e63f97580cf985 P da72d89bb77b471bbbdd19123c4776a9: stopping tablet replica
I20260810 07:15:17.479527 25055 tablet_replica.cc:330] T 0327cbdf92324218b5e63f97580cf985 P dc01a69a8f784435b348edba6864ce55: stopping tablet replica
I20260810 07:15:17.479557 25056 raft_consensus.cc:2229] T 0327cbdf92324218b5e63f97580cf985 P da72d89bb77b471bbbdd19123c4776a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:17.479583 25055 raft_consensus.cc:2229] T 0327cbdf92324218b5e63f97580cf985 P dc01a69a8f784435b348edba6864ce55 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:15:17.479596 25056 raft_consensus.cc:2258] T 0327cbdf92324218b5e63f97580cf985 P da72d89bb77b471bbbdd19123c4776a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:17.479626 25055 raft_consensus.cc:2258] T 0327cbdf92324218b5e63f97580cf985 P dc01a69a8f784435b348edba6864ce55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:17.479722 24773 tablet_service.cc:1514] Processing DeleteTablet for tablet 0327cbdf92324218b5e63f97580cf985 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:17 UTC) from {username='oryx', principal='oryx/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:49630
I20260810 07:15:17.479863 25057 tablet_replica.cc:330] T 0327cbdf92324218b5e63f97580cf985 P 0b2b56cd5f6d4f64be7f73b6a75c439d: stopping tablet replica
I20260810 07:15:17.479915 25057 raft_consensus.cc:2229] T 0327cbdf92324218b5e63f97580cf985 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:17.479952 25057 raft_consensus.cc:2258] T 0327cbdf92324218b5e63f97580cf985 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:17.479966 25056 ts_tablet_manager.cc:1905] T 0327cbdf92324218b5e63f97580cf985 P da72d89bb77b471bbbdd19123c4776a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:17.480347 25055 ts_tablet_manager.cc:1905] T 0327cbdf92324218b5e63f97580cf985 P dc01a69a8f784435b348edba6864ce55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:17.480309 24469 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41292:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:15:17.480473 24469 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:15:17.481633 25056 ts_tablet_manager.cc:1918] T 0327cbdf92324218b5e63f97580cf985 P da72d89bb77b471bbbdd19123c4776a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:15:17.481702 25056 log.cc:1196] T 0327cbdf92324218b5e63f97580cf985 P da72d89bb77b471bbbdd19123c4776a9: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-2/wal/wals/0327cbdf92324218b5e63f97580cf985
I20260810 07:15:17.481719 25055 ts_tablet_manager.cc:1918] T 0327cbdf92324218b5e63f97580cf985 P dc01a69a8f784435b348edba6864ce55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:15:17.481794 25055 log.cc:1196] T 0327cbdf92324218b5e63f97580cf985 P dc01a69a8f784435b348edba6864ce55: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-0/wal/wals/0327cbdf92324218b5e63f97580cf985
I20260810 07:15:17.481942 25056 ts_tablet_manager.cc:1939] T 0327cbdf92324218b5e63f97580cf985 P da72d89bb77b471bbbdd19123c4776a9: Deleting consensus metadata
I20260810 07:15:17.482060 25055 ts_tablet_manager.cc:1939] T 0327cbdf92324218b5e63f97580cf985 P dc01a69a8f784435b348edba6864ce55: Deleting consensus metadata
I20260810 07:15:17.482285 24452 catalog_manager.cc:4841] TS da72d89bb77b471bbbdd19123c4776a9 (127.15.217.67:40883): tablet 0327cbdf92324218b5e63f97580cf985 (table test-table [id=4dbad75d0d494faaa57d8b7a39daa3cd]) successfully deleted
I20260810 07:15:17.482399 24456 catalog_manager.cc:4841] TS dc01a69a8f784435b348edba6864ce55 (127.15.217.65:41485): tablet 0327cbdf92324218b5e63f97580cf985 (table test-table [id=4dbad75d0d494faaa57d8b7a39daa3cd]) successfully deleted
I20260810 07:15:17.482756 25057 ts_tablet_manager.cc:1905] T 0327cbdf92324218b5e63f97580cf985 P 0b2b56cd5f6d4f64be7f73b6a75c439d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:17.483546 24622 tablet_service.cc:1467] Processing CreateTablet for tablet e1f1cfa236dc47bea7fd86ab92475b00 (DEFAULT_TABLE table=test-table [id=aaf3d04c77f64e4c8f5441dcc42adfbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:17.483592 24932 tablet_service.cc:1467] Processing CreateTablet for tablet e1f1cfa236dc47bea7fd86ab92475b00 (DEFAULT_TABLE table=test-table [id=aaf3d04c77f64e4c8f5441dcc42adfbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:17.483659 24622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1f1cfa236dc47bea7fd86ab92475b00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:17.483697 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1f1cfa236dc47bea7fd86ab92475b00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:17.484383 25057 ts_tablet_manager.cc:1918] T 0327cbdf92324218b5e63f97580cf985 P 0b2b56cd5f6d4f64be7f73b6a75c439d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:15:17.484439 25057 log.cc:1196] T 0327cbdf92324218b5e63f97580cf985 P 0b2b56cd5f6d4f64be7f73b6a75c439d: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-1/wal/wals/0327cbdf92324218b5e63f97580cf985
I20260810 07:15:17.484687 24773 tablet_service.cc:1467] Processing CreateTablet for tablet e1f1cfa236dc47bea7fd86ab92475b00 (DEFAULT_TABLE table=test-table [id=aaf3d04c77f64e4c8f5441dcc42adfbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:17.484815 24773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1f1cfa236dc47bea7fd86ab92475b00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:17.485713 25032 tablet_bootstrap.cc:492] T e1f1cfa236dc47bea7fd86ab92475b00 P 0b2b56cd5f6d4f64be7f73b6a75c439d: Bootstrap starting.
I20260810 07:15:17.485739 25033 tablet_bootstrap.cc:492] T e1f1cfa236dc47bea7fd86ab92475b00 P da72d89bb77b471bbbdd19123c4776a9: Bootstrap starting.
I20260810 07:15:17.486064 25032 tablet_bootstrap.cc:654] T e1f1cfa236dc47bea7fd86ab92475b00 P 0b2b56cd5f6d4f64be7f73b6a75c439d: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:17.486143 25033 tablet_bootstrap.cc:654] T e1f1cfa236dc47bea7fd86ab92475b00 P da72d89bb77b471bbbdd19123c4776a9: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:17.486841 25033 tablet_bootstrap.cc:492] T e1f1cfa236dc47bea7fd86ab92475b00 P da72d89bb77b471bbbdd19123c4776a9: No bootstrap required, opened a new log
I20260810 07:15:17.486891 25033 ts_tablet_manager.cc:1397] T e1f1cfa236dc47bea7fd86ab92475b00 P da72d89bb77b471bbbdd19123c4776a9: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 07:15:17.487020 25033 raft_consensus.cc:348] T e1f1cfa236dc47bea7fd86ab92475b00 P da72d89bb77b471bbbdd19123c4776a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } }
I20260810 07:15:17.487085 25033 raft_consensus.cc:374] T e1f1cfa236dc47bea7fd86ab92475b00 P da72d89bb77b471bbbdd19123c4776a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:17.487104 25033 raft_consensus.cc:729] T e1f1cfa236dc47bea7fd86ab92475b00 P da72d89bb77b471bbbdd19123c4776a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da72d89bb77b471bbbdd19123c4776a9, State: Initialized, Role: FOLLOWER
I20260810 07:15:17.487145 25033 consensus_queue.cc:260] T e1f1cfa236dc47bea7fd86ab92475b00 P da72d89bb77b471bbbdd19123c4776a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } }
I20260810 07:15:17.487192 25031 tablet_bootstrap.cc:492] T e1f1cfa236dc47bea7fd86ab92475b00 P dc01a69a8f784435b348edba6864ce55: Bootstrap starting.
I20260810 07:15:17.487228 25033 ts_tablet_manager.cc:1428] T e1f1cfa236dc47bea7fd86ab92475b00 P da72d89bb77b471bbbdd19123c4776a9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:17.484701 25057 ts_tablet_manager.cc:1939] T 0327cbdf92324218b5e63f97580cf985 P 0b2b56cd5f6d4f64be7f73b6a75c439d: Deleting consensus metadata
I20260810 07:15:17.487498 25031 tablet_bootstrap.cc:654] T e1f1cfa236dc47bea7fd86ab92475b00 P dc01a69a8f784435b348edba6864ce55: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:17.487699 24451 catalog_manager.cc:4841] TS 0b2b56cd5f6d4f64be7f73b6a75c439d (127.15.217.66:46469): tablet 0327cbdf92324218b5e63f97580cf985 (table test-table [id=4dbad75d0d494faaa57d8b7a39daa3cd]) successfully deleted
I20260810 07:15:17.488019 25032 tablet_bootstrap.cc:492] T e1f1cfa236dc47bea7fd86ab92475b00 P 0b2b56cd5f6d4f64be7f73b6a75c439d: No bootstrap required, opened a new log
I20260810 07:15:17.488080 25032 ts_tablet_manager.cc:1397] T e1f1cfa236dc47bea7fd86ab92475b00 P 0b2b56cd5f6d4f64be7f73b6a75c439d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:15:17.488190 25032 raft_consensus.cc:348] T e1f1cfa236dc47bea7fd86ab92475b00 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } }
I20260810 07:15:17.488283 25032 raft_consensus.cc:374] T e1f1cfa236dc47bea7fd86ab92475b00 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:17.488312 25032 raft_consensus.cc:729] T e1f1cfa236dc47bea7fd86ab92475b00 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b2b56cd5f6d4f64be7f73b6a75c439d, State: Initialized, Role: FOLLOWER
I20260810 07:15:17.488361 25032 consensus_queue.cc:260] T e1f1cfa236dc47bea7fd86ab92475b00 P 0b2b56cd5f6d4f64be7f73b6a75c439d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } }
I20260810 07:15:17.488480 25032 ts_tablet_manager.cc:1428] T e1f1cfa236dc47bea7fd86ab92475b00 P 0b2b56cd5f6d4f64be7f73b6a75c439d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:17.488718 25031 tablet_bootstrap.cc:492] T e1f1cfa236dc47bea7fd86ab92475b00 P dc01a69a8f784435b348edba6864ce55: No bootstrap required, opened a new log
I20260810 07:15:17.488801 25031 ts_tablet_manager.cc:1397] T e1f1cfa236dc47bea7fd86ab92475b00 P dc01a69a8f784435b348edba6864ce55: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 07:15:17.488919 25031 raft_consensus.cc:348] T e1f1cfa236dc47bea7fd86ab92475b00 P dc01a69a8f784435b348edba6864ce55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } }
I20260810 07:15:17.488993 25031 raft_consensus.cc:374] T e1f1cfa236dc47bea7fd86ab92475b00 P dc01a69a8f784435b348edba6864ce55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:17.489019 25031 raft_consensus.cc:729] T e1f1cfa236dc47bea7fd86ab92475b00 P dc01a69a8f784435b348edba6864ce55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc01a69a8f784435b348edba6864ce55, State: Initialized, Role: FOLLOWER
I20260810 07:15:17.489075 25031 consensus_queue.cc:260] T e1f1cfa236dc47bea7fd86ab92475b00 P dc01a69a8f784435b348edba6864ce55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } }
I20260810 07:15:17.489177 25031 ts_tablet_manager.cc:1428] T e1f1cfa236dc47bea7fd86ab92475b00 P dc01a69a8f784435b348edba6864ce55: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 07:15:17.620982 25063 raft_consensus.cc:482] T e1f1cfa236dc47bea7fd86ab92475b00 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:15:17.621107 25063 raft_consensus.cc:504] T e1f1cfa236dc47bea7fd86ab92475b00 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } }
I20260810 07:15:17.621341 25063 leader_election.cc:288] T e1f1cfa236dc47bea7fd86ab92475b00 P 0b2b56cd5f6d4f64be7f73b6a75c439d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc01a69a8f784435b348edba6864ce55 (127.15.217.65:41485), da72d89bb77b471bbbdd19123c4776a9 (127.15.217.67:40883)
I20260810 07:15:17.621553 24642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1f1cfa236dc47bea7fd86ab92475b00" candidate_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc01a69a8f784435b348edba6864ce55" is_pre_election: true
I20260810 07:15:17.621558 24954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1f1cfa236dc47bea7fd86ab92475b00" candidate_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da72d89bb77b471bbbdd19123c4776a9" is_pre_election: true
I20260810 07:15:17.621701 24642 raft_consensus.cc:2454] T e1f1cfa236dc47bea7fd86ab92475b00 P dc01a69a8f784435b348edba6864ce55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b2b56cd5f6d4f64be7f73b6a75c439d in term 0.
I20260810 07:15:17.621694 24954 raft_consensus.cc:2454] T e1f1cfa236dc47bea7fd86ab92475b00 P da72d89bb77b471bbbdd19123c4776a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b2b56cd5f6d4f64be7f73b6a75c439d in term 0.
I20260810 07:15:17.621886 24723 leader_election.cc:302] T e1f1cfa236dc47bea7fd86ab92475b00 P 0b2b56cd5f6d4f64be7f73b6a75c439d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b2b56cd5f6d4f64be7f73b6a75c439d, dc01a69a8f784435b348edba6864ce55; no voters:
I20260810 07:15:17.622007 25063 raft_consensus.cc:2781] T e1f1cfa236dc47bea7fd86ab92475b00 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:15:17.622066 25063 raft_consensus.cc:482] T e1f1cfa236dc47bea7fd86ab92475b00 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:15:17.622082 25063 raft_consensus.cc:3037] T e1f1cfa236dc47bea7fd86ab92475b00 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:17.622929 25063 raft_consensus.cc:504] T e1f1cfa236dc47bea7fd86ab92475b00 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } }
I20260810 07:15:17.623122 25063 leader_election.cc:288] T e1f1cfa236dc47bea7fd86ab92475b00 P 0b2b56cd5f6d4f64be7f73b6a75c439d [CANDIDATE]: Term 1 election: Requested vote from peers dc01a69a8f784435b348edba6864ce55 (127.15.217.65:41485), da72d89bb77b471bbbdd19123c4776a9 (127.15.217.67:40883)
I20260810 07:15:17.623294 24642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1f1cfa236dc47bea7fd86ab92475b00" candidate_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc01a69a8f784435b348edba6864ce55"
I20260810 07:15:17.623304 24954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1f1cfa236dc47bea7fd86ab92475b00" candidate_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da72d89bb77b471bbbdd19123c4776a9"
I20260810 07:15:17.623392 24642 raft_consensus.cc:3037] T e1f1cfa236dc47bea7fd86ab92475b00 P dc01a69a8f784435b348edba6864ce55 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:17.623401 24954 raft_consensus.cc:3037] T e1f1cfa236dc47bea7fd86ab92475b00 P da72d89bb77b471bbbdd19123c4776a9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:17.624173 24954 raft_consensus.cc:2454] T e1f1cfa236dc47bea7fd86ab92475b00 P da72d89bb77b471bbbdd19123c4776a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b2b56cd5f6d4f64be7f73b6a75c439d in term 1.
I20260810 07:15:17.624185 24642 raft_consensus.cc:2454] T e1f1cfa236dc47bea7fd86ab92475b00 P dc01a69a8f784435b348edba6864ce55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b2b56cd5f6d4f64be7f73b6a75c439d in term 1.
I20260810 07:15:17.624357 24721 leader_election.cc:302] T e1f1cfa236dc47bea7fd86ab92475b00 P 0b2b56cd5f6d4f64be7f73b6a75c439d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b2b56cd5f6d4f64be7f73b6a75c439d, da72d89bb77b471bbbdd19123c4776a9; no voters:
I20260810 07:15:17.624495 25063 raft_consensus.cc:2781] T e1f1cfa236dc47bea7fd86ab92475b00 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:17.624552 25063 raft_consensus.cc:686] T e1f1cfa236dc47bea7fd86ab92475b00 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 1 LEADER]: Becoming Leader. State: Replica: 0b2b56cd5f6d4f64be7f73b6a75c439d, State: Running, Role: LEADER
I20260810 07:15:17.624607 25063 consensus_queue.cc:237] T e1f1cfa236dc47bea7fd86ab92475b00 P 0b2b56cd5f6d4f64be7f73b6a75c439d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } }
I20260810 07:15:17.625289 24469 catalog_manager.cc:5495] T e1f1cfa236dc47bea7fd86ab92475b00 P 0b2b56cd5f6d4f64be7f73b6a75c439d reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b2b56cd5f6d4f64be7f73b6a75c439d (127.15.217.66). New cstate: current_term: 1 leader_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b2b56cd5f6d4f64be7f73b6a75c439d" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 } health_report { overall_health: UNKNOWN } } }
I20260810 07:15:17.637641 25029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:17.635300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49648 (local address 127.15.217.66:46469)
0810 07:15:17.635344 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:17.635347 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:17.635355 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:15:17.635414 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:17.635415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:17.635425 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:17.635480 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:15:17.635608 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.636090 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.636925 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.637131 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.637224 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:17.637372 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:15:17.637409 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:15:17.637480 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:17.637537 (+ 57us) server_negotiation.cc:299] Negotiation successful
0810 07:15:17.637559 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 07:15:17.639842 24642 raft_consensus.cc:1261] T e1f1cfa236dc47bea7fd86ab92475b00 P dc01a69a8f784435b348edba6864ce55 [term 1 FOLLOWER]: Refusing update from remote peer 0b2b56cd5f6d4f64be7f73b6a75c439d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:17.640009 24954 raft_consensus.cc:1261] T e1f1cfa236dc47bea7fd86ab92475b00 P da72d89bb77b471bbbdd19123c4776a9 [term 1 FOLLOWER]: Refusing update from remote peer 0b2b56cd5f6d4f64be7f73b6a75c439d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:17.640113 25085 consensus_queue.cc:1035] T e1f1cfa236dc47bea7fd86ab92475b00 P 0b2b56cd5f6d4f64be7f73b6a75c439d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc01a69a8f784435b348edba6864ce55" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:17.640386 25085 consensus_queue.cc:1035] T e1f1cfa236dc47bea7fd86ab92475b00 P 0b2b56cd5f6d4f64be7f73b6a75c439d [LEADER]: Connected to new peer: Peer: permanent_uuid: "da72d89bb77b471bbbdd19123c4776a9" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:17.650592 24469 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41292:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:17.650699 24469 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41292:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:17.651341 24469 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0b336a4b1aac4e909a20ccd2a426c4a7: Sending DeleteTablet for 3 replicas of tablet e1f1cfa236dc47bea7fd86ab92475b00
I20260810 07:15:17.651610 24622 tablet_service.cc:1514] Processing DeleteTablet for tablet e1f1cfa236dc47bea7fd86ab92475b00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:17 UTC) from {username='oryx', principal='oryx/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:57402
I20260810 07:15:17.651643 24932 tablet_service.cc:1514] Processing DeleteTablet for tablet e1f1cfa236dc47bea7fd86ab92475b00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:17 UTC) from {username='oryx', principal='oryx/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:55606
I20260810 07:15:17.651628 24774 tablet_service.cc:1514] Processing DeleteTablet for tablet e1f1cfa236dc47bea7fd86ab92475b00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:17 UTC) from {username='oryx', principal='oryx/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:49630
I20260810 07:15:17.651726 25056 tablet_replica.cc:330] T e1f1cfa236dc47bea7fd86ab92475b00 P da72d89bb77b471bbbdd19123c4776a9: stopping tablet replica
I20260810 07:15:17.651738 25055 tablet_replica.cc:330] T e1f1cfa236dc47bea7fd86ab92475b00 P dc01a69a8f784435b348edba6864ce55: stopping tablet replica
I20260810 07:15:17.651744 25057 tablet_replica.cc:330] T e1f1cfa236dc47bea7fd86ab92475b00 P 0b2b56cd5f6d4f64be7f73b6a75c439d: stopping tablet replica
I20260810 07:15:17.651772 25056 raft_consensus.cc:2229] T e1f1cfa236dc47bea7fd86ab92475b00 P da72d89bb77b471bbbdd19123c4776a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:17.651798 25056 raft_consensus.cc:2258] T e1f1cfa236dc47bea7fd86ab92475b00 P da72d89bb77b471bbbdd19123c4776a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:17.651798 25057 raft_consensus.cc:2229] T e1f1cfa236dc47bea7fd86ab92475b00 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:15:17.651798 25055 raft_consensus.cc:2229] T e1f1cfa236dc47bea7fd86ab92475b00 P dc01a69a8f784435b348edba6864ce55 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:17.651834 25055 raft_consensus.cc:2258] T e1f1cfa236dc47bea7fd86ab92475b00 P dc01a69a8f784435b348edba6864ce55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:17.651860 25057 raft_consensus.cc:2258] T e1f1cfa236dc47bea7fd86ab92475b00 P 0b2b56cd5f6d4f64be7f73b6a75c439d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:17.652197 25057 ts_tablet_manager.cc:1905] T e1f1cfa236dc47bea7fd86ab92475b00 P 0b2b56cd5f6d4f64be7f73b6a75c439d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:17.652204 25055 ts_tablet_manager.cc:1905] T e1f1cfa236dc47bea7fd86ab92475b00 P dc01a69a8f784435b348edba6864ce55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:17.652326 25056 ts_tablet_manager.cc:1905] T e1f1cfa236dc47bea7fd86ab92475b00 P da72d89bb77b471bbbdd19123c4776a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:17.654153 25056 ts_tablet_manager.cc:1918] T e1f1cfa236dc47bea7fd86ab92475b00 P da72d89bb77b471bbbdd19123c4776a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:15:17.654151 25055 ts_tablet_manager.cc:1918] T e1f1cfa236dc47bea7fd86ab92475b00 P dc01a69a8f784435b348edba6864ce55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:15:17.654186 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 24550
I20260810 07:15:17.654228 25055 log.cc:1196] T e1f1cfa236dc47bea7fd86ab92475b00 P dc01a69a8f784435b348edba6864ce55: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-0/wal/wals/e1f1cfa236dc47bea7fd86ab92475b00
I20260810 07:15:17.654229 25056 log.cc:1196] T e1f1cfa236dc47bea7fd86ab92475b00 P da72d89bb77b471bbbdd19123c4776a9: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-2/wal/wals/e1f1cfa236dc47bea7fd86ab92475b00
I20260810 07:15:17.654452 25056 ts_tablet_manager.cc:1939] T e1f1cfa236dc47bea7fd86ab92475b00 P da72d89bb77b471bbbdd19123c4776a9: Deleting consensus metadata
I20260810 07:15:17.654471 25055 ts_tablet_manager.cc:1939] T e1f1cfa236dc47bea7fd86ab92475b00 P dc01a69a8f784435b348edba6864ce55: Deleting consensus metadata
I20260810 07:15:17.656693 25057 ts_tablet_manager.cc:1918] T e1f1cfa236dc47bea7fd86ab92475b00 P 0b2b56cd5f6d4f64be7f73b6a75c439d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:15:17.656754 25057 log.cc:1196] T e1f1cfa236dc47bea7fd86ab92475b00 P 0b2b56cd5f6d4f64be7f73b6a75c439d: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346081759552-16229-0/minicluster-data/ts-1/wal/wals/e1f1cfa236dc47bea7fd86ab92475b00
I20260810 07:15:17.656985 25057 ts_tablet_manager.cc:1939] T e1f1cfa236dc47bea7fd86ab92475b00 P 0b2b56cd5f6d4f64be7f73b6a75c439d: Deleting consensus metadata
I20260810 07:15:17.657330 24451 catalog_manager.cc:4841] TS 0b2b56cd5f6d4f64be7f73b6a75c439d (127.15.217.66:46469): tablet e1f1cfa236dc47bea7fd86ab92475b00 (table test-table [id=aaf3d04c77f64e4c8f5441dcc42adfbd]) successfully deleted
I20260810 07:15:17.658120 24452 catalog_manager.cc:4841] TS da72d89bb77b471bbbdd19123c4776a9 (127.15.217.67:40883): tablet e1f1cfa236dc47bea7fd86ab92475b00 (table test-table [id=aaf3d04c77f64e4c8f5441dcc42adfbd]) successfully deleted
W20260810 07:15:17.664363 24456 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.217.65:41485 (error 108)
I20260810 07:15:17.664813 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 24702
I20260810 07:15:17.667909 25119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:17.664841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:41485 (local address 127.0.0.1:57434)
0810 07:15:17.664960 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:17.667867 (+ 2907us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.65:41485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":43,"threads_started":1}
W20260810 07:15:17.668077 24456 catalog_manager.cc:4584] TS dc01a69a8f784435b348edba6864ce55 (127.15.217.65:41485): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e1f1cfa236dc47bea7fd86ab92475b00: Network error: Client connection negotiation failed: client connection to 127.15.217.65:41485: connect: Connection refused (error 111)
I20260810 07:15:17.673302 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 24855
I20260810 07:15:17.679684 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 24434
2026-08-10T07:15:17Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2630 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:15:17 dist-test-slave-10pc krb5kdc[25122](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:15:17 dist-test-slave-10pc krb5kdc[25122](info): set up 2 sockets
Aug 10 07:15:17 dist-test-slave-10pc krb5kdc[25122](info): commencing operation
krb5kdc: starting...
W20260810 07:15:18.780802 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:15:18 dist-test-slave-10pc krb5kdc[25122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346118, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:15:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:15:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
I20260810 07:15:18.915189 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:44645
--webserver_interface=127.15.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:36951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:44645
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:19.001037 25136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:19.001297 25136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:19.001339 25136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:19.002599 25136 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:19.002668 25136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:19.002694 25136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:19.002727 25136 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:19.002753 25136 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:15:19.004771 25136 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:36951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:44645
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:44645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:19.005183 25136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:19.005471 25136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:19.010567 25145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:19.010720 25143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:19.010807 25136 server_base.cc:1034] running on GCE node
W20260810 07:15:19.010727 25142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:19.011219 25136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:19.011554 25136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:19.012732 25136 hybrid_clock.cc:648] HybridClock initialized: now 1786346119012724 us; error 37 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-10T07:15:19Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1342 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:15:19 dist-test-slave-10pc krb5kdc[25150](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:15:19 dist-test-slave-10pc krb5kdc[25150](info): set up 2 sockets
Aug 10 07:15:19 dist-test-slave-10pc krb5kdc[25150](info): commencing operation
krb5kdc: starting...
W20260810 07:15:20.117355 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:15:20 dist-test-slave-10pc krb5kdc[25150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:15:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:15:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
I20260810 07:15:20.253381 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:42079
--webserver_interface=127.15.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:43307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:42079
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:20.341187 25164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:20.341418 25164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:20.341450 25164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:20.342669 25164 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:20.342720 25164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:20.342734 25164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:20.342747 25164 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:20.342759 25164 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:15:20.344286 25164 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:43307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:42079
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:42079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:20.344636 25164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:20.344874 25164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:20.350205 25172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:20.350211 25170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:20.350240 25169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:20.350417 25164 server_base.cc:1034] running on GCE node
I20260810 07:15:20.350761 25164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:20.351069 25164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:20.352240 25164 hybrid_clock.cc:648] HybridClock initialized: now 1786346120352235 us; error 56 us; skew 500 ppm
Aug 10 07:15:20 dist-test-slave-10pc krb5kdc[25150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346120, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:20.354871 25164 init.cc:377] Logged in from keytab as kudu/127.15.217.126@KRBTEST.COM (short username kudu)
I20260810 07:15:20.355394 25164 webserver.cc:460] Webserver started at http://127.15.217.126:46615/ using document root <none> and password file <none>
I20260810 07:15:20.355633 25164 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:20.355702 25164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:20.355849 25164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:20.357021 25164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "131c79f77f2642ad9e0a16b65f9fbb26"
format_stamp: "Formatted at 2026-08-10 07:15:20 on dist-test-slave-10pc"
server_key: "264989e7c59de2ba5dbdb50f154e8a26"
server_key_iv: "564a9bdcb37eeeee8a7168ab0382e19e"
server_key_version: "encryptionkey@0"
I20260810 07:15:20.357402 25164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "131c79f77f2642ad9e0a16b65f9fbb26"
format_stamp: "Formatted at 2026-08-10 07:15:20 on dist-test-slave-10pc"
server_key: "264989e7c59de2ba5dbdb50f154e8a26"
server_key_iv: "564a9bdcb37eeeee8a7168ab0382e19e"
server_key_version: "encryptionkey@0"
I20260810 07:15:20.358753 25164 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:15:20.359539 25180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:20.359714 25164 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 07:15:20.359828 25164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "131c79f77f2642ad9e0a16b65f9fbb26"
format_stamp: "Formatted at 2026-08-10 07:15:20 on dist-test-slave-10pc"
server_key: "264989e7c59de2ba5dbdb50f154e8a26"
server_key_iv: "564a9bdcb37eeeee8a7168ab0382e19e"
server_key_version: "encryptionkey@0"
I20260810 07:15:20.359899 25164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:20.367677 25164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:20.369825 25164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:20.369942 25164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:20.373736 25164 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:42079
I20260810 07:15:20.373804 25234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:42079 every 8 connection(s)
I20260810 07:15:20.374164 25164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:15:20.375067 25235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:20.377517 25235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26: Bootstrap starting.
I20260810 07:15:20.378139 25235 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:20.378458 25235 log.cc:824] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:20.379223 25235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26: No bootstrap required, opened a new log
I20260810 07:15:20.379562 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 25164
I20260810 07:15:20.379662 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:15:20.379851 16229 external_mini_cluster.cc:1428] Setting key 0c63a3cdefb7c89077979f253f64a00c
I20260810 07:15:20.381629 25235 raft_consensus.cc:348] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "131c79f77f2642ad9e0a16b65f9fbb26" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42079 } }
I20260810 07:15:20.381816 25235 raft_consensus.cc:374] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:20.381861 25235 raft_consensus.cc:729] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 131c79f77f2642ad9e0a16b65f9fbb26, State: Initialized, Role: FOLLOWER
I20260810 07:15:20.382022 25235 consensus_queue.cc:260] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "131c79f77f2642ad9e0a16b65f9fbb26" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42079 } }
I20260810 07:15:20.382092 25235 raft_consensus.cc:388] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:15:20.382115 25235 raft_consensus.cc:482] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:15:20.382200 25235 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:20.383388 25235 raft_consensus.cc:504] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "131c79f77f2642ad9e0a16b65f9fbb26" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42079 } }
I20260810 07:15:20.383503 25235 leader_election.cc:302] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 131c79f77f2642ad9e0a16b65f9fbb26; no voters:
I20260810 07:15:20.383662 25235 leader_election.cc:288] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:15:20.383725 25241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:20.383915 25235 sys_catalog.cc:564] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:20.383939 25241 raft_consensus.cc:686] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26 [term 1 LEADER]: Becoming Leader. State: Replica: 131c79f77f2642ad9e0a16b65f9fbb26, State: Running, Role: LEADER
I20260810 07:15:20.384147 25241 consensus_queue.cc:237] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "131c79f77f2642ad9e0a16b65f9fbb26" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42079 } }
I20260810 07:15:20.384635 25242 sys_catalog.cc:455] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "131c79f77f2642ad9e0a16b65f9fbb26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "131c79f77f2642ad9e0a16b65f9fbb26" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42079 } } }
I20260810 07:15:20.384716 25242 sys_catalog.cc:458] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:20.385185 25251 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:20.385192 25243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 131c79f77f2642ad9e0a16b65f9fbb26. Latest consensus state: current_term: 1 leader_uuid: "131c79f77f2642ad9e0a16b65f9fbb26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "131c79f77f2642ad9e0a16b65f9fbb26" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42079 } } }
I20260810 07:15:20.385565 25243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:20.385793 25251 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:15:20.387838 25251 catalog_manager.cc:1347] Generated new cluster ID: e3126fc108e240e69edf5577d8a55448
I20260810 07:15:20.387897 25251 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 07:15:20 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:15:20.392278 25240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:20.380438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42802 (local address 127.15.217.126:42079)
0810 07:15:20.380693 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:20.380703 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:20.380738 (+ 35us) server_negotiation.cc:407] Connection header received
0810 07:15:20.381587 (+ 849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:20.381598 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:20.381675 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:20.381832 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:20.382701 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:20.384211 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:20.385480 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:20.386099 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:20.386222 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:20.388886 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:20.388891 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:20.388891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:20.388904 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:20.390667 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:20.391073 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:20.391080 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:20.391081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:20.391198 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:20.391425 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:20.391425 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:20.391426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:20.391684 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:20.391853 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:20.391961 (+ 108us) server_negotiation.cc:299] Negotiation successful
0810 07:15:20.392113 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":112,"threads_started":1}
I20260810 07:15:20.434481 25251 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:15:20.435277 25251 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:20.442116 25251 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26: Generated new TSK 0
I20260810 07:15:20.442338 25251 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab.
Entry for principal kudu/127.15.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab.
I20260810 07:15:20.478828 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.65:0
--local_ip_for_outbound_sockets=127.15.217.65
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:42079
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:43307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:20.575318 25264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:20.575562 25264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:20.575589 25264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:20.576942 25264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:20.576996 25264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:20.577036 25264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.65
I20260810 07:15:20.578713 25264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:43307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:42079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.65
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:20.579011 25264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:20.579278 25264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:20.585019 25270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:20.585029 25271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:20.585050 25274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:20.585443 25264 server_base.cc:1034] running on GCE node
I20260810 07:15:20.585698 25264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:20.585958 25264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:20.587138 25264 hybrid_clock.cc:648] HybridClock initialized: now 1786346120587125 us; error 32 us; skew 500 ppm
Aug 10 07:15:20 dist-test-slave-10pc krb5kdc[25150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346120, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:20.590003 25264 init.cc:377] Logged in from keytab as kudu/127.15.217.65@KRBTEST.COM (short username kudu)
I20260810 07:15:20.590507 25264 webserver.cc:460] Webserver started at http://127.15.217.65:35107/ using document root <none> and password file <none>
I20260810 07:15:20.590795 25264 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:20.590876 25264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:20.591049 25264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:20.592270 25264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-0/data/instance:
uuid: "9b5fa2f356d6411b89b56f54a3b1d99c"
format_stamp: "Formatted at 2026-08-10 07:15:20 on dist-test-slave-10pc"
server_key: "65ac695c28ea8d8b95397dabea072f97"
server_key_iv: "d9202c305db3033e6c4b9e5ecc9c0794"
server_key_version: "encryptionkey@0"
I20260810 07:15:20.592736 25264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance:
uuid: "9b5fa2f356d6411b89b56f54a3b1d99c"
format_stamp: "Formatted at 2026-08-10 07:15:20 on dist-test-slave-10pc"
server_key: "65ac695c28ea8d8b95397dabea072f97"
server_key_iv: "d9202c305db3033e6c4b9e5ecc9c0794"
server_key_version: "encryptionkey@0"
I20260810 07:15:20.594273 25264 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:20.595201 25281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:20.595397 25264 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:20.595515 25264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-0/wal
uuid: "9b5fa2f356d6411b89b56f54a3b1d99c"
format_stamp: "Formatted at 2026-08-10 07:15:20 on dist-test-slave-10pc"
server_key: "65ac695c28ea8d8b95397dabea072f97"
server_key_iv: "d9202c305db3033e6c4b9e5ecc9c0794"
server_key_version: "encryptionkey@0"
I20260810 07:15:20.595606 25264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:20.622143 25264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:20.624540 25264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:20.624754 25264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:20.625056 25264 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:20.625427 25264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:20.625509 25264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:20.625550 25264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:20.625574 25264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:20.632324 25264 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.65:39911
I20260810 07:15:20.632387 25400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.65:39911 every 8 connection(s)
I20260810 07:15:20.632755 25264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
I20260810 07:15:20.635790 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 25264
I20260810 07:15:20.635892 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance
I20260810 07:15:20.636096 16229 external_mini_cluster.cc:1428] Setting key 4f86437602c0a7a1bf135781c02d05bd
Aug 10 07:15:20 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346120, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:15:20.643733 25403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:20.633441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.65:40111)
0810 07:15:20.633610 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:20.633658 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:20.634213 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:20.634488 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:20.634493 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:20.635092 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:20.635311 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:20.635316 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:20.636351 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:20.636353 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:20.637424 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:20.637427 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:20.637957 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:20.637958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:20.638078 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:20.638659 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:20.638670 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:20.640497 (+ 1827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:20.642205 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:20.642214 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:20.642216 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:20.642538 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:20.642776 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:20.642777 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:20.642778 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:20.642897 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:20.643210 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:20.643215 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:20.643337 (+ 122us) client_negotiation.cc:769] Sending connection context
0810 07:15:20.643423 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 07:15:20.643552 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":75,"threads_started":1}
I20260810 07:15:20.643728 25240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:20.633488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:40111 (local address 127.15.217.126:42079)
0810 07:15:20.633540 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:20.633543 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:20.633754 (+ 211us) server_negotiation.cc:407] Connection header received
0810 07:15:20.634329 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:20.634330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:20.634347 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:20.634408 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:20.635424 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:20.636212 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:20.637516 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:20.637869 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:20.637944 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:20.640592 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:20.640593 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:20.640593 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:20.640602 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:20.642116 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:20.642622 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:20.642623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:20.642624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:20.642696 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:20.642968 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:20.642970 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:20.642970 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:20.643135 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:20.643216 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:20.643475 (+ 259us) server_negotiation.cc:299] Negotiation successful
0810 07:15:20.643591 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 07:15:20.644446 25401 heartbeater.cc:347] Connected to a master server at 127.15.217.126:42079
I20260810 07:15:20.644587 25401 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:20.644799 25401 heartbeater.cc:511] Master 127.15.217.126:42079 requested a full tablet report, sending...
I20260810 07:15:20.645368 25197 ts_manager.cc:194] Registered new tserver with Master: 9b5fa2f356d6411b89b56f54a3b1d99c (127.15.217.65:39911)
I20260810 07:15:20.646100 25197 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.65@KRBTEST.COM'} at 127.15.217.65:40111
WARNING: no policy specified for kudu/127.15.217.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab.
Entry for principal kudu/127.15.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab.
I20260810 07:15:20.665196 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.66:0
--local_ip_for_outbound_sockets=127.15.217.66
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:42079
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:43307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:20.753190 25406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:20.753407 25406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:20.753432 25406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:20.754629 25406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:20.754674 25406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:20.754701 25406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.66
I20260810 07:15:20.756222 25406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:43307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:42079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.66
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:20.756520 25406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:20.756779 25406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:20.762035 25415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:20.762086 25413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:20.762228 25406 server_base.cc:1034] running on GCE node
W20260810 07:15:20.762259 25412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:20.762528 25406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:20.762830 25406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:20.763998 25406 hybrid_clock.cc:648] HybridClock initialized: now 1786346120763974 us; error 59 us; skew 500 ppm
Aug 10 07:15:20 dist-test-slave-10pc krb5kdc[25150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346120, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:20.766731 25406 init.cc:377] Logged in from keytab as kudu/127.15.217.66@KRBTEST.COM (short username kudu)
I20260810 07:15:20.767153 25406 webserver.cc:460] Webserver started at http://127.15.217.66:42847/ using document root <none> and password file <none>
I20260810 07:15:20.767351 25406 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:20.767393 25406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:20.767484 25406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:20.768615 25406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-1/data/instance:
uuid: "3eccf6907e514580b9e172e778edbc1b"
format_stamp: "Formatted at 2026-08-10 07:15:20 on dist-test-slave-10pc"
server_key: "994114c57543f81e63d092e513ec45d0"
server_key_iv: "e2ebfb3429e29b0e76c85c0a023a6c91"
server_key_version: "encryptionkey@0"
I20260810 07:15:20.768952 25406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance:
uuid: "3eccf6907e514580b9e172e778edbc1b"
format_stamp: "Formatted at 2026-08-10 07:15:20 on dist-test-slave-10pc"
server_key: "994114c57543f81e63d092e513ec45d0"
server_key_iv: "e2ebfb3429e29b0e76c85c0a023a6c91"
server_key_version: "encryptionkey@0"
I20260810 07:15:20.770326 25406 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:15:20.771116 25423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:20.771310 25406 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:20.771409 25406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-1/wal
uuid: "3eccf6907e514580b9e172e778edbc1b"
format_stamp: "Formatted at 2026-08-10 07:15:20 on dist-test-slave-10pc"
server_key: "994114c57543f81e63d092e513ec45d0"
server_key_iv: "e2ebfb3429e29b0e76c85c0a023a6c91"
server_key_version: "encryptionkey@0"
I20260810 07:15:20.771471 25406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:20.785413 25406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:20.787896 25406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:20.788031 25406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:20.788234 25406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:20.788664 25406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:20.788714 25406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:20.788738 25406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:20.788784 25406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:20.794881 25406 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.66:39159
I20260810 07:15:20.794963 25544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.66:39159 every 8 connection(s)
I20260810 07:15:20.795342 25406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
I20260810 07:15:20.801321 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 25406
I20260810 07:15:20.801468 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance
I20260810 07:15:20.801710 16229 external_mini_cluster.cc:1428] Setting key b36b3eef5f69d23449fab8cf39c66ffa
Aug 10 07:15:20 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346120, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:15:20.805869 25240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:20.796177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:36737 (local address 127.15.217.126:42079)
0810 07:15:20.796268 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:20.796270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:20.796502 (+ 232us) server_negotiation.cc:407] Connection header received
0810 07:15:20.797077 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:20.797078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:20.797090 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:20.797151 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:20.798229 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:20.798837 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:20.799922 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:20.800142 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:20.800200 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:20.802709 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:20.802710 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:20.802711 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:20.802718 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:20.804090 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:20.804672 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:20.804674 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:20.804675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:20.804749 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:20.805084 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:20.805085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:20.805085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:20.805260 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:20.805333 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:20.805638 (+ 305us) server_negotiation.cc:299] Negotiation successful
0810 07:15:20.805738 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 07:15:20.805888 25548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:20.796134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.66:36737)
0810 07:15:20.796330 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:20.796370 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:20.796900 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:20.797238 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:20.797242 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:20.797921 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:20.798134 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:20.798140 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:20.798937 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:20.798938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:20.799844 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:20.799847 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:20.800222 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:20.800223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:20.800327 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:20.800826 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:20.800835 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:20.802616 (+ 1781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:20.804178 (+ 1562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:20.804187 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:20.804190 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:20.804572 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:20.804843 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:20.804844 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:20.804845 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:20.804995 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:20.805333 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:20.805341 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:20.805486 (+ 145us) client_negotiation.cc:769] Sending connection context
0810 07:15:20.805576 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 07:15:20.805713 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":105,"threads_started":1}
I20260810 07:15:20.806473 25545 heartbeater.cc:347] Connected to a master server at 127.15.217.126:42079
I20260810 07:15:20.806598 25545 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:20.806792 25545 heartbeater.cc:511] Master 127.15.217.126:42079 requested a full tablet report, sending...
I20260810 07:15:20.807233 25197 ts_manager.cc:194] Registered new tserver with Master: 3eccf6907e514580b9e172e778edbc1b (127.15.217.66:39159)
I20260810 07:15:20.807632 25197 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.66@KRBTEST.COM'} at 127.15.217.66:36737
WARNING: no policy specified for kudu/127.15.217.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab.
Entry for principal kudu/127.15.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab.
I20260810 07:15:20.830888 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.67:0
--local_ip_for_outbound_sockets=127.15.217.67
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:42079
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:43307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:20.917594 25551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:20.917820 25551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:20.917845 25551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:20.918967 25551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:20.919003 25551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:20.919028 25551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.67
I20260810 07:15:20.920439 25551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:43307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:42079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.67
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:20.920719 25551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:20.920920 25551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:20.925967 25560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:20.925961 25556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:20.925961 25558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:20.926384 25551 server_base.cc:1034] running on GCE node
I20260810 07:15:20.926638 25551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:20.926901 25551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:20.928072 25551 hybrid_clock.cc:648] HybridClock initialized: now 1786346120928051 us; error 39 us; skew 500 ppm
Aug 10 07:15:20 dist-test-slave-10pc krb5kdc[25150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346120, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:20.930739 25551 init.cc:377] Logged in from keytab as kudu/127.15.217.67@KRBTEST.COM (short username kudu)
I20260810 07:15:20.931241 25551 webserver.cc:460] Webserver started at http://127.15.217.67:36911/ using document root <none> and password file <none>
I20260810 07:15:20.931504 25551 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:20.931576 25551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:20.931730 25551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:20.932921 25551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-2/data/instance:
uuid: "7ca8ae4bd3ba4e49b625c761eaeb9019"
format_stamp: "Formatted at 2026-08-10 07:15:20 on dist-test-slave-10pc"
server_key: "b486f694ee6097c6761638ae9862a288"
server_key_iv: "b3aa11d521a7e7e6211cef0ea8a9be95"
server_key_version: "encryptionkey@0"
I20260810 07:15:20.933306 25551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance:
uuid: "7ca8ae4bd3ba4e49b625c761eaeb9019"
format_stamp: "Formatted at 2026-08-10 07:15:20 on dist-test-slave-10pc"
server_key: "b486f694ee6097c6761638ae9862a288"
server_key_iv: "b3aa11d521a7e7e6211cef0ea8a9be95"
server_key_version: "encryptionkey@0"
I20260810 07:15:20.934718 25551 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260810 07:15:20.935487 25567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:20.935700 25551 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:20.935813 25551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-2/wal
uuid: "7ca8ae4bd3ba4e49b625c761eaeb9019"
format_stamp: "Formatted at 2026-08-10 07:15:20 on dist-test-slave-10pc"
server_key: "b486f694ee6097c6761638ae9862a288"
server_key_iv: "b3aa11d521a7e7e6211cef0ea8a9be95"
server_key_version: "encryptionkey@0"
I20260810 07:15:20.935892 25551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:20.951414 25551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:20.953522 25551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:20.953703 25551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:20.953959 25551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:20.954306 25551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:20.954363 25551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:20.954399 25551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:20.954447 25551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:20.960911 25551 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.67:46497
I20260810 07:15:20.960966 25689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.67:46497 every 8 connection(s)
I20260810 07:15:20.961440 25551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
I20260810 07:15:20.967149 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 25551
I20260810 07:15:20.967257 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance
I20260810 07:15:20.967446 16229 external_mini_cluster.cc:1428] Setting key 9eacdcbec44abdec5c3c1284b24888a2
Aug 10 07:15:20 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346120, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:15:20.971983 25240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:20.962389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:36613 (local address 127.15.217.126:42079)
0810 07:15:20.962460 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:20.962462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:20.962694 (+ 232us) server_negotiation.cc:407] Connection header received
0810 07:15:20.963273 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:20.963275 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:20.963287 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:20.963343 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:20.964467 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:20.965007 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:20.966066 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:20.966282 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:20.966362 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:20.969178 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:20.969180 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:20.969180 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:20.969188 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:20.970513 (+ 1325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:20.970952 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:20.970953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:20.970953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:20.971015 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:20.971287 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:20.971288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:20.971289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:20.971412 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:20.971494 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:20.971788 (+ 294us) server_negotiation.cc:299] Negotiation successful
0810 07:15:20.971876 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 07:15:20.972172 25692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:20.962383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.67:36613)
0810 07:15:20.962545 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:20.962595 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:20.963143 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:20.963430 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:20.963434 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:20.964117 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:20.964337 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:20.964344 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:20.965126 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:20.965127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:20.965953 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:20.965955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:20.966375 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:20.966376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:20.966516 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:20.967084 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:20.967097 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:20.969076 (+ 1979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:20.970620 (+ 1544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:20.970628 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:20.970631 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:20.970868 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:20.971089 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:20.971090 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:20.971091 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:20.971218 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:20.971485 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:20.971493 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:20.971643 (+ 150us) client_negotiation.cc:769] Sending connection context
0810 07:15:20.971752 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 07:15:20.971923 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":80,"threads_started":1}
I20260810 07:15:20.972748 25690 heartbeater.cc:347] Connected to a master server at 127.15.217.126:42079
I20260810 07:15:20.972862 25690 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:20.973026 25690 heartbeater.cc:511] Master 127.15.217.126:42079 requested a full tablet report, sending...
I20260810 07:15:20.973443 25197 ts_manager.cc:194] Registered new tserver with Master: 7ca8ae4bd3ba4e49b625c761eaeb9019 (127.15.217.67:46497)
I20260810 07:15:20.973819 25197 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.67@KRBTEST.COM'} at 127.15.217.67:36613
I20260810 07:15:20.979319 16229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.15.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.65.keytab.
Entry for principal oryx/127.15.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.65.keytab.
WARNING: no policy specified for oryx/127.15.217.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.217.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.66.keytab.
Entry for principal oryx/127.15.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.66.keytab.
WARNING: no policy specified for oryx/127.15.217.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.217.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.67.keytab.
Entry for principal oryx/127.15.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.67.keytab.
I20260810 07:15:21.063591 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 25264
I20260810 07:15:21.076727 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 25406
I20260810 07:15:21.082952 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 25551
I20260810 07:15:21.089468 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 25164
I20260810 07:15:21.097344 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:42079
--webserver_interface=127.15.217.126
--webserver_port=46615
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:43307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:42079
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:21.182854 25699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:21.183094 25699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:21.183120 25699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:21.184360 25699 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:21.184406 25699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:21.184420 25699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:21.184433 25699 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:21.184521 25699 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:15:21.186103 25699 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:43307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:42079
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:42079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=46615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:21.186396 25699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:21.186643 25699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:21.191612 25704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:21.191605 25705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:21.191605 25707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:21.191753 25699 server_base.cc:1034] running on GCE node
I20260810 07:15:21.192137 25699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:21.192489 25699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:21.193657 25699 hybrid_clock.cc:648] HybridClock initialized: now 1786346121193646 us; error 29 us; skew 500 ppm
Aug 10 07:15:21 dist-test-slave-10pc krb5kdc[25150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346121, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:21.196218 25699 init.cc:377] Logged in from keytab as kudu/127.15.217.126@KRBTEST.COM (short username kudu)
I20260810 07:15:21.196714 25699 webserver.cc:460] Webserver started at http://127.15.217.126:46615/ using document root <none> and password file <none>
I20260810 07:15:21.196925 25699 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:21.196978 25699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:21.198300 25699 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:21.198987 25715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:21.199209 25699 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:21.199311 25699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "131c79f77f2642ad9e0a16b65f9fbb26"
format_stamp: "Formatted at 2026-08-10 07:15:20 on dist-test-slave-10pc"
server_key: "264989e7c59de2ba5dbdb50f154e8a26"
server_key_iv: "564a9bdcb37eeeee8a7168ab0382e19e"
server_key_version: "encryptionkey@0"
I20260810 07:15:21.199723 25699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:21.209932 25699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:21.212355 25699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:21.212548 25699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:21.216636 25699 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:42079
I20260810 07:15:21.216703 25769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:42079 every 8 connection(s)
I20260810 07:15:21.217060 25699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:15:21.218565 25770 sys_catalog.cc:263] Verifying existing consensus state
I20260810 07:15:21.219275 25770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26: Bootstrap starting.
I20260810 07:15:21.223029 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 25699
I20260810 07:15:21.223348 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.65:39911
--local_ip_for_outbound_sockets=127.15.217.65
--tserver_master_addrs=127.15.217.126:42079
--webserver_port=35107
--webserver_interface=127.15.217.65
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:43307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.217.65
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:15:21.229534 25770 log.cc:824] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:21.232448 25770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:15:21.232854 25770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26: Bootstrap complete.
I20260810 07:15:21.235447 25770 raft_consensus.cc:348] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "131c79f77f2642ad9e0a16b65f9fbb26" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42079 } }
I20260810 07:15:21.235710 25770 raft_consensus.cc:729] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 131c79f77f2642ad9e0a16b65f9fbb26, State: Initialized, Role: FOLLOWER
I20260810 07:15:21.235889 25770 consensus_queue.cc:260] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "131c79f77f2642ad9e0a16b65f9fbb26" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42079 } }
I20260810 07:15:21.235966 25770 raft_consensus.cc:388] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:15:21.236017 25770 raft_consensus.cc:482] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:15:21.236065 25770 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26 [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:15:21.237314 25770 raft_consensus.cc:504] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "131c79f77f2642ad9e0a16b65f9fbb26" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42079 } }
I20260810 07:15:21.237455 25770 leader_election.cc:302] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 131c79f77f2642ad9e0a16b65f9fbb26; no voters:
I20260810 07:15:21.237675 25770 leader_election.cc:288] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 07:15:21.237717 25774 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 07:15:21.237859 25774 raft_consensus.cc:686] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26 [term 2 LEADER]: Becoming Leader. State: Replica: 131c79f77f2642ad9e0a16b65f9fbb26, State: Running, Role: LEADER
I20260810 07:15:21.237969 25770 sys_catalog.cc:564] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:21.237950 25774 consensus_queue.cc:237] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "131c79f77f2642ad9e0a16b65f9fbb26" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42079 } }
I20260810 07:15:21.238355 25774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 131c79f77f2642ad9e0a16b65f9fbb26. Latest consensus state: current_term: 2 leader_uuid: "131c79f77f2642ad9e0a16b65f9fbb26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "131c79f77f2642ad9e0a16b65f9fbb26" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42079 } } }
I20260810 07:15:21.238442 25774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:21.238723 25776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "131c79f77f2642ad9e0a16b65f9fbb26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "131c79f77f2642ad9e0a16b65f9fbb26" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42079 } } }
I20260810 07:15:21.238829 25776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:21.240021 25792 catalog_manager.cc:1259] Loaded cluster ID: e3126fc108e240e69edf5577d8a55448
I20260810 07:15:21.240111 25792 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26: loading cluster ID for follower catalog manager: success
I20260810 07:15:21.241433 25792 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26: acquiring CA information for follower catalog manager: success
I20260810 07:15:21.241863 25792 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:15:21.242040 25780 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:21.242275 25780 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:15:21.242349 25780 catalog_manager.cc:1259] Loaded cluster ID: e3126fc108e240e69edf5577d8a55448
I20260810 07:15:21.242381 25780 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:15:21.242599 25780 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:21.242708 25780 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 131c79f77f2642ad9e0a16b65f9fbb26: Loaded TSK: 0
I20260810 07:15:21.242851 25780 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 07:15:21.314322 25772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:21.314571 25772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:21.314616 25772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:21.315858 25772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:21.315932 25772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:21.315977 25772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.65
I20260810 07:15:21.317539 25772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:43307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.65.keytab
--principal=oryx/127.15.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.65:39911
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.217.65
--webserver_port=35107
--tserver_master_addrs=127.15.217.126:42079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.65
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:21.317814 25772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:21.318086 25772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:21.324383 25801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:21.324393 25799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:21.324374 25798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:21.324646 25772 server_base.cc:1034] running on GCE node
I20260810 07:15:21.325145 25772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:21.325446 25772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:21.326637 25772 hybrid_clock.cc:648] HybridClock initialized: now 1786346121326623 us; error 28 us; skew 500 ppm
Aug 10 07:15:21 dist-test-slave-10pc krb5kdc[25150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346121, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:21.329648 25772 init.cc:377] Logged in from keytab as oryx/127.15.217.65@KRBTEST.COM (short username oryx)
I20260810 07:15:21.330265 25772 webserver.cc:460] Webserver started at http://127.15.217.65:35107/ using document root <none> and password file <none>
I20260810 07:15:21.330535 25772 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:21.330628 25772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:21.332134 25772 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 07:15:21.332865 25808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:21.333040 25772 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 07:15:21.333158 25772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-0/wal
uuid: "9b5fa2f356d6411b89b56f54a3b1d99c"
format_stamp: "Formatted at 2026-08-10 07:15:20 on dist-test-slave-10pc"
server_key: "65ac695c28ea8d8b95397dabea072f97"
server_key_iv: "d9202c305db3033e6c4b9e5ecc9c0794"
server_key_version: "encryptionkey@0"
I20260810 07:15:21.333554 25772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:21.346848 25772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:21.349313 25772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:21.349480 25772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:21.349781 25772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:21.350201 25772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:21.350242 25772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:21.350266 25772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:21.350278 25772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:21.357908 25772 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.65:39911
I20260810 07:15:21.358006 25928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.65:39911 every 8 connection(s)
I20260810 07:15:21.358374 25772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
I20260810 07:15:21.360277 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 25772
I20260810 07:15:21.360654 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.66:39159
--local_ip_for_outbound_sockets=127.15.217.66
--tserver_master_addrs=127.15.217.126:42079
--webserver_port=42847
--webserver_interface=127.15.217.66
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:43307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.217.66
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 07:15:21 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:21 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:21.367882 25931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:21.359146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.65:53955)
0810 07:15:21.359361 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:21.359411 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:21.360075 (+ 664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:21.360619 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:21.360623 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:21.361378 (+ 755us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:21.361643 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:21.361649 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.363054 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.363056 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:21.363870 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:21.363873 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.364568 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.364569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:21.364709 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:21.365288 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:21.365297 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:21.367655 (+ 2358us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":96,"threads_started":1}
W20260810 07:15:21.368134 25929 heartbeater.cc:644] Failed to heartbeat to 127.15.217.126:42079 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.217.126:42079: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
W20260810 07:15:21.368183 25933 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:21.359273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:53955 (local address 127.15.217.126:42079)
0810 07:15:21.359468 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:21.359477 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:21.359507 (+ 30us) server_negotiation.cc:407] Connection header received
0810 07:15:21.360310 (+ 803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:21.360318 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:21.360387 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:21.360574 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:21.361798 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.362938 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.363995 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.364403 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.364580 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:21.367781 (+ 3201us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.65:53955: BlockingRecv error: recv got EOF from 127.15.217.65:53955 (error 108)
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":80,"threads_started":1}
Aug 10 07:15:21 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:21 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:21.372852 25931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:21.368377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.65:53365)
0810 07:15:21.368441 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:21.368500 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:21.368553 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:21.368700 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:21.368701 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:21.368835 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:21.368883 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:21.368884 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.369518 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.369519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:21.370020 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:21.370022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.370376 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.370377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:21.370457 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:21.370911 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:21.370916 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:21.372764 (+ 1848us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 07:15:21.372905 25933 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:21.368518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:53365 (local address 127.15.217.126:42079)
0810 07:15:21.368555 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:21.368557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:21.368565 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:15:21.368612 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:21.368612 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:21.368623 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:21.368698 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:21.368967 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.369432 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.370104 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.370303 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.370378 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:21.372843 (+ 2465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.65:53365: BlockingRecv error: recv got EOF from 127.15.217.65:53365 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Aug 10 07:15:21 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:21 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:21.378018 25931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:21.373167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.65:59145)
0810 07:15:21.373272 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:21.373286 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:21.373349 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:21.373498 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:21.373499 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:21.373661 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:21.373702 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:21.373705 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.374329 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.374330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:21.375150 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:21.375152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.375490 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.375491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:21.375550 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:21.376015 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:21.376020 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:21.377945 (+ 1925us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":85}
W20260810 07:15:21.378176 25933 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:21.373219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:59145 (local address 127.15.217.126:42079)
0810 07:15:21.373279 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:21.373281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:21.373344 (+ 63us) server_negotiation.cc:407] Connection header received
0810 07:15:21.373413 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:21.373414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:21.373422 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:21.373471 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:21.373782 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.374246 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.375240 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.375414 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.375470 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:21.378032 (+ 2562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.65:59145: BlockingRecv error: recv got EOF from 127.15.217.65:59145 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260810 07:15:21.378161 25929 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 07:15:21.452406 25934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:21.452713 25934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:21.452739 25934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:21.453876 25934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:21.453910 25934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:21.453935 25934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.66
I20260810 07:15:21.455310 25934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:43307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.66.keytab
--principal=oryx/127.15.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.66:39159
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.217.66
--webserver_port=42847
--tserver_master_addrs=127.15.217.126:42079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.66
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:21.455497 25934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:21.455681 25934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:21.460862 25939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:21.460888 25942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:21.460855 25940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:21.461208 25934 server_base.cc:1034] running on GCE node
I20260810 07:15:21.461437 25934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:21.461699 25934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:21.462855 25934 hybrid_clock.cc:648] HybridClock initialized: now 1786346121462849 us; error 36 us; skew 500 ppm
Aug 10 07:15:21 dist-test-slave-10pc krb5kdc[25150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346121, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:21.465687 25934 init.cc:377] Logged in from keytab as oryx/127.15.217.66@KRBTEST.COM (short username oryx)
I20260810 07:15:21.466193 25934 webserver.cc:460] Webserver started at http://127.15.217.66:42847/ using document root <none> and password file <none>
I20260810 07:15:21.466435 25934 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:21.466517 25934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:21.468034 25934 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 07:15:21.468930 25949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:21.469158 25934 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:15:21.469259 25934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-1/wal
uuid: "3eccf6907e514580b9e172e778edbc1b"
format_stamp: "Formatted at 2026-08-10 07:15:20 on dist-test-slave-10pc"
server_key: "994114c57543f81e63d092e513ec45d0"
server_key_iv: "e2ebfb3429e29b0e76c85c0a023a6c91"
server_key_version: "encryptionkey@0"
I20260810 07:15:21.469833 25934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:21.491253 25934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:21.493718 25934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:21.493901 25934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:21.494179 25934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:21.494550 25934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:21.494607 25934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:21.494645 25934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:21.494692 25934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:21.500888 25934 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.66:39159
I20260810 07:15:21.500941 26072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.66:39159 every 8 connection(s)
I20260810 07:15:21.501317 25934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
Aug 10 07:15:21 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:21.507999 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 25934
I20260810 07:15:21.508322 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.67:46497
--local_ip_for_outbound_sockets=127.15.217.67
--tserver_master_addrs=127.15.217.126:42079
--webserver_port=36911
--webserver_interface=127.15.217.67
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:43307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.217.67
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 07:15:21 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:21.509264 26075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:21.502080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.66:35125)
0810 07:15:21.502257 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:21.502297 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:21.502952 (+ 655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:21.503222 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:21.503226 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:21.503878 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:21.504095 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:21.504101 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.504876 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.504877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:21.505714 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:21.505717 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.506116 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.506117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:21.506243 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:21.506808 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:21.506817 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:21.509110 (+ 2293us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":88,"threads_started":1}
W20260810 07:15:21.509330 25933 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:21.502087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:35125 (local address 127.15.217.126:42079)
0810 07:15:21.502152 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:21.502154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:21.502390 (+ 236us) server_negotiation.cc:407] Connection header received
0810 07:15:21.503065 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:21.503066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:21.503084 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:21.503134 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:21.504199 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.504774 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.505810 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.506014 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.506092 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:21.509243 (+ 3151us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.66:35125: BlockingRecv error: recv got EOF from 127.15.217.66:35125 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260810 07:15:21.509646 26073 heartbeater.cc:644] Failed to heartbeat to 127.15.217.126:42079 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.217.126:42079: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Aug 10 07:15:21 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:21 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:21.514820 26075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:21.509916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.66:39655)
0810 07:15:21.509962 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:21.509974 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:21.510009 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:21.510263 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:21.510265 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:21.510438 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:21.510506 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:21.510508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.511394 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.511395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:21.511901 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:21.511903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.512289 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.512290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:21.512367 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:21.512874 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:21.512879 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:21.514768 (+ 1889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260810 07:15:21.514986 25933 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:21.509970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:39655 (local address 127.15.217.126:42079)
0810 07:15:21.510125 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:21.510127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:21.510136 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:15:21.510168 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:21.510169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:21.510178 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:21.510248 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:21.510584 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.511307 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.512002 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.512207 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.512291 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:21.514897 (+ 2606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.66:39655: BlockingRecv error: recv got EOF from 127.15.217.66:39655 (error 108)
Metrics: {"server-negotiator.queue_time_us":130}
Aug 10 07:15:21 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:21 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:21.519914 26075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:21.515175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.66:51115)
0810 07:15:21.515220 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:21.515229 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:21.515270 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:21.515409 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:21.515410 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:21.515516 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:21.515545 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:21.515546 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.516517 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.516518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:21.516994 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:21.516995 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.517485 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.517486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:21.517524 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:21.517997 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:21.518001 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:21.519864 (+ 1863us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260810 07:15:21.520097 25933 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:21.515228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:51115 (local address 127.15.217.126:42079)
0810 07:15:21.515265 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:21.515267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:21.515276 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:15:21.515322 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:21.515323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:21.515335 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:21.515392 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:21.515616 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.516397 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.517075 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.517403 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.517482 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:21.520015 (+ 2533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.66:51115: BlockingRecv error: recv got EOF from 127.15.217.66:51115 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260810 07:15:21.520205 26073 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 07:15:21.596554 26076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:21.596798 26076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:21.596840 26076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:21.598089 26076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:21.598155 26076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:21.598201 26076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.67
I20260810 07:15:21.599664 26076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:43307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/krb5kdc/oryx_127.15.217.67.keytab
--principal=oryx/127.15.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.67:46497
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.217.67
--webserver_port=36911
--tserver_master_addrs=127.15.217.126:42079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.67
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:21.599936 26076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:21.600167 26076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:21.605557 26084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:21.605611 26081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:21.605600 26082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:21.605769 26076 server_base.cc:1034] running on GCE node
I20260810 07:15:21.605994 26076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:21.606256 26076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:21.607434 26076 hybrid_clock.cc:648] HybridClock initialized: now 1786346121607408 us; error 35 us; skew 500 ppm
Aug 10 07:15:21 dist-test-slave-10pc krb5kdc[25150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346121, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:21.610191 26076 init.cc:377] Logged in from keytab as oryx/127.15.217.67@KRBTEST.COM (short username oryx)
I20260810 07:15:21.610679 26076 webserver.cc:460] Webserver started at http://127.15.217.67:36911/ using document root <none> and password file <none>
I20260810 07:15:21.610929 26076 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:21.611023 26076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:21.612519 26076 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:15:21.613198 26091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:21.613374 26076 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:21.613480 26076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-2/wal
uuid: "7ca8ae4bd3ba4e49b625c761eaeb9019"
format_stamp: "Formatted at 2026-08-10 07:15:20 on dist-test-slave-10pc"
server_key: "b486f694ee6097c6761638ae9862a288"
server_key_iv: "b3aa11d521a7e7e6211cef0ea8a9be95"
server_key_version: "encryptionkey@0"
I20260810 07:15:21.613897 26076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:21.628052 26076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:21.630366 26076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:21.630573 26076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:21.630836 26076 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:21.631197 26076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:21.631259 26076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:21.631312 26076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:21.631359 26076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:21.637887 26076 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.67:46497
I20260810 07:15:21.637935 26216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.67:46497 every 8 connection(s)
I20260810 07:15:21.638373 26076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
Aug 10 07:15:21 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:21 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:21.645681 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 26076
W20260810 07:15:21.646452 25933 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:21.639137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:37149 (local address 127.15.217.126:42079)
0810 07:15:21.639191 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:21.639193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:21.639469 (+ 276us) server_negotiation.cc:407] Connection header received
0810 07:15:21.640080 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:21.640081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:21.640097 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:21.640155 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:21.641240 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.641750 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.642820 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.643044 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.643119 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:21.646370 (+ 3251us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.67:37149: BlockingRecv error: recv got EOF from 127.15.217.67:37149 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 07:15:21.646705 26220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:21.639128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.67:37149)
0810 07:15:21.639321 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:21.639359 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:21.639932 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:21.640229 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:21.640234 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:21.640916 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:21.641125 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:21.641130 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.641853 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.641854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:21.642717 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:21.642719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.643153 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.643154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:21.643284 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:21.643832 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:21.643841 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:21.646402 (+ 2561us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":115,"threads_started":1}
W20260810 07:15:21.646927 26217 heartbeater.cc:644] Failed to heartbeat to 127.15.217.126:42079 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.217.126:42079: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Aug 10 07:15:21 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:21 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:21.651708 26220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:21.647181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.67:49377)
0810 07:15:21.647274 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:21.647299 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:21.647354 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:21.647458 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:21.647458 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:21.647609 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:21.647657 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:21.647659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.648340 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.648340 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:21.648954 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:21.648956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.649296 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.649297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:21.649366 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:21.649804 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:21.649808 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:21.651631 (+ 1823us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":71}
W20260810 07:15:21.651814 25933 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:21.647228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:49377 (local address 127.15.217.126:42079)
0810 07:15:21.647271 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:21.647273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:21.647339 (+ 66us) server_negotiation.cc:407] Connection header received
0810 07:15:21.647405 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:21.647406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:21.647414 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:21.647453 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:21.647729 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.648258 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.649038 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.649222 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.649275 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:21.651742 (+ 2467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.67:49377: BlockingRecv error: recv got EOF from 127.15.217.67:49377 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 07:15:21.654464 26222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:21.646274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42814 (local address 127.15.217.126:42079)
0810 07:15:21.646393 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:21.646395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:21.646404 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:15:21.646430 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:21.646430 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:21.646441 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:21.646515 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:21.647329 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.647982 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.648953 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.649239 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.649328 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:21.650725 (+ 1397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:21.650732 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:21.650732 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:21.650745 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:21.652900 (+ 2155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:21.653313 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:21.653317 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:21.653317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:21.653447 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:21.653650 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:21.653651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:21.653652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:21.653972 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:21.654133 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:21.654255 (+ 122us) server_negotiation.cc:299] Negotiation successful
0810 07:15:21.654345 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":16,"server-negotiator.queue_time_us":97,"thread_start_us":73,"threads_started":1}
Aug 10 07:15:21 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:21 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:21.656765 26220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:21.651965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.67:48533)
0810 07:15:21.652097 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:21.652107 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:21.652155 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:21.652291 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:21.652293 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:21.652414 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:21.652442 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:21.652443 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.653212 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.653212 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:21.653982 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:21.653984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.654484 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.654485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:21.654558 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:21.654963 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:21.654967 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:21.656720 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116}
W20260810 07:15:21.656895 25933 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:21.652056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:48533 (local address 127.15.217.126:42079)
0810 07:15:21.652116 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:21.652118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:21.652151 (+ 33us) server_negotiation.cc:407] Connection header received
0810 07:15:21.652193 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:21.652194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:21.652202 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:21.652269 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:21.652586 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.653124 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.654065 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.654363 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.654428 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:21.656820 (+ 2392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.67:48533: BlockingRecv error: recv got EOF from 127.15.217.67:48533 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260810 07:15:21.656906 26217 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 10 07:15:22 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:22 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:22.384044 26223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:22.378652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.65:53743)
0810 07:15:22.378827 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:22.378844 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:22.378916 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:22.379095 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:22.379096 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:22.379283 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:22.379341 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:22.379343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:22.380085 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:22.380086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:22.380792 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:22.380796 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:22.381434 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:22.381435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:22.381513 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:22.382061 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:22.382066 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:22.383944 (+ 1878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":77,"threads_started":1}
W20260810 07:15:22.384166 26224 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:22.378729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:53743 (local address 127.15.217.126:42079)
0810 07:15:22.378900 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:22.378903 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:22.378913 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:15:22.378970 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:22.378971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:22.378981 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:22.379072 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:22.379451 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:22.379976 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:22.380917 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:22.381336 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:22.381432 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:22.384084 (+ 2652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.65:53743: BlockingRecv error: recv got EOF from 127.15.217.65:53743 (error 108)
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":78,"threads_started":1}
Aug 10 07:15:22 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:22 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:22.525661 26225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:22.520667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.66:57943)
0810 07:15:22.520837 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:22.520853 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:22.520935 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:22.521095 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:22.521096 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:22.521272 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:22.521329 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:22.521331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:22.521960 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:22.521960 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:22.522532 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:22.522534 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:22.522861 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:22.522862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:22.522905 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:22.523350 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:22.523373 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:22.525587 (+ 2214us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":88,"threads_started":1}
W20260810 07:15:22.525774 26224 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:22.520730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:57943 (local address 127.15.217.126:42079)
0810 07:15:22.520822 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:22.520824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:22.520925 (+ 101us) server_negotiation.cc:407] Connection header received
0810 07:15:22.521008 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:22.521009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:22.521021 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:22.521070 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:22.521392 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:22.521891 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:22.522603 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:22.522795 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:22.522845 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:22.525697 (+ 2852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.66:57943: BlockingRecv error: recv got EOF from 127.15.217.66:57943 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 10 07:15:22 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:22 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:22.662700 26226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:22.657299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.67:60915)
0810 07:15:22.657483 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:22.657502 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:22.657597 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:22.657771 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:22.657772 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:22.657971 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:22.658061 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:22.658063 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:22.658747 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:22.658749 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:22.659333 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:22.659334 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:22.659722 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:22.659723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:22.659789 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:22.660609 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:22.660616 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:22.662594 (+ 1978us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":94,"threads_started":1}
W20260810 07:15:22.662774 26224 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:22.657391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:60915 (local address 127.15.217.126:42079)
0810 07:15:22.657467 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:22.657469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:22.657572 (+ 103us) server_negotiation.cc:407] Connection header received
0810 07:15:22.657674 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:22.657675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:22.657691 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:22.657762 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:22.658149 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:22.658660 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:22.659423 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:22.659632 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:22.659718 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:22.662705 (+ 2987us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.67:60915: BlockingRecv error: recv got EOF from 127.15.217.67:60915 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 10 07:15:23 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:23 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:23.389580 26227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:23.384644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.65:33733)
0810 07:15:23.384809 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:23.384828 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:23.384884 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:23.385053 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:23.385055 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:23.385245 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:23.385304 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:23.385306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:23.385970 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:23.385971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:23.386535 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:23.386537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:23.386947 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:23.386948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:23.387006 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:23.387527 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:23.387532 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:23.389512 (+ 1980us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":99,"threads_started":1}
W20260810 07:15:23.389729 26228 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:23.384737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:33733 (local address 127.15.217.126:42079)
0810 07:15:23.384876 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:23.384878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:23.384894 (+ 16us) server_negotiation.cc:407] Connection header received
0810 07:15:23.384955 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:23.384956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:23.384965 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:23.385031 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:23.385377 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:23.385892 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:23.386618 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:23.386845 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:23.386900 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:23.389624 (+ 2724us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.65:33733: BlockingRecv error: recv got EOF from 127.15.217.65:33733 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":83,"threads_started":1}
Aug 10 07:15:23 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:23 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:23.531212 26229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:23.526174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.66:35463)
0810 07:15:23.526334 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:23.526353 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:23.526414 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:23.526564 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:23.526565 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:23.526739 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:23.526795 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:23.526796 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:23.527502 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:23.527503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:23.528048 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:23.528050 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:23.528437 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:23.528439 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:23.528561 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:23.529105 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:23.529110 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:23.531147 (+ 2037us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":100,"threads_started":1}
W20260810 07:15:23.531395 26228 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:23.526268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:35463 (local address 127.15.217.126:42079)
0810 07:15:23.526334 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:23.526336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:23.526418 (+ 82us) server_negotiation.cc:407] Connection header received
0810 07:15:23.526470 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:23.526470 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:23.526482 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:23.526542 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:23.526862 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:23.527414 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:23.528137 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:23.528355 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:23.528431 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:23.531321 (+ 2890us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.66:35463: BlockingRecv error: recv got EOF from 127.15.217.66:35463 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 10 07:15:23 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:23 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:23.668004 26230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:23.663250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.67:45885)
0810 07:15:23.663450 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:23.663463 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:23.663523 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:23.663678 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:23.663680 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:23.663846 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:23.663900 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:23.663902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:23.664630 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:23.664631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:23.665182 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:23.665183 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:23.665578 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:23.665579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:23.665647 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:23.666110 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:23.666115 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:23.667911 (+ 1796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":85,"threads_started":1}
W20260810 07:15:23.668182 26228 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:23.663458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:45885 (local address 127.15.217.126:42079)
0810 07:15:23.663520 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:23.663522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:23.663533 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:15:23.663590 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:23.663590 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:23.663603 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:23.663655 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:23.663965 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:23.664530 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:23.665275 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:23.665496 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:23.665577 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:23.668048 (+ 2471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.67:45885: BlockingRecv error: recv got EOF from 127.15.217.67:45885 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 10 07:15:24 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:24 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:24.394901 26231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:24.390158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.65:60997)
0810 07:15:24.390351 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:24.390364 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:24.390443 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:24.390574 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:24.390575 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:24.390747 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:24.390803 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:24.390804 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:24.391434 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:24.391435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:24.391994 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:24.391996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:24.392330 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:24.392331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:24.392375 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:24.392877 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:24.392881 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:24.394833 (+ 1952us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":109,"threads_started":1}
W20260810 07:15:24.395123 26232 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:24.390301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:60997 (local address 127.15.217.126:42079)
0810 07:15:24.390438 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:24.390440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:24.390451 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:15:24.390505 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:24.390505 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:24.390516 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:24.390567 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:24.390871 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:24.391371 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:24.392071 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:24.392258 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:24.392308 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:24.394979 (+ 2671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.65:60997: BlockingRecv error: recv got EOF from 127.15.217.65:60997 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":75,"threads_started":1}
Aug 10 07:15:24 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:24 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:24.537263 26233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:24.531842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.66:45071)
0810 07:15:24.532045 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:24.532059 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:24.532158 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:24.532359 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:24.532360 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:24.532615 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:24.532674 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:24.532676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:24.533390 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:24.533392 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:24.533952 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:24.533954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:24.534341 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:24.534342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:24.534404 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:24.534894 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:24.534899 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:24.537160 (+ 2261us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":105,"threads_started":1}
W20260810 07:15:24.537333 26232 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:24.531957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:45071 (local address 127.15.217.126:42079)
0810 07:15:24.532042 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:24.532045 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:24.532134 (+ 89us) server_negotiation.cc:407] Connection header received
0810 07:15:24.532241 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:24.532242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:24.532256 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:24.532327 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:24.532760 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:24.533294 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:24.534045 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:24.534259 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:24.534341 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:24.537267 (+ 2926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.66:45071: BlockingRecv error: recv got EOF from 127.15.217.66:45071 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 10 07:15:24 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:24 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:24.673890 26234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:24.668642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.67:59015)
0810 07:15:24.668809 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:24.668828 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:24.668896 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:24.669061 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:24.669062 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:24.669268 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:24.669320 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:24.669322 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:24.669977 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:24.669978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:24.670537 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:24.670538 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:24.670945 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:24.670945 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:24.670988 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:24.671506 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:24.671512 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:24.673827 (+ 2315us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":112,"threads_started":1}
W20260810 07:15:24.674055 26232 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:24.668705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:59015 (local address 127.15.217.126:42079)
0810 07:15:24.668792 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:24.668794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:24.668924 (+ 130us) server_negotiation.cc:407] Connection header received
0810 07:15:24.668960 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:24.668960 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:24.668976 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:24.669044 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:24.669394 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:24.669907 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:24.670639 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:24.670867 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:24.670938 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:24.673974 (+ 3036us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.67:59015: BlockingRecv error: recv got EOF from 127.15.217.67:59015 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 10 07:15:25 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:25 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:25.400777 26235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:25.395564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.65:45883)
0810 07:15:25.395747 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:25.395761 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:25.395838 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:25.395988 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:25.395988 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:25.396166 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:25.396224 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:25.396226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:25.396953 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:25.396955 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:25.397534 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:25.397537 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:25.397961 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:25.397961 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:25.398009 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:25.398524 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:25.398529 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:25.400715 (+ 2186us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":86,"threads_started":1}
W20260810 07:15:25.400957 26236 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:25.395635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:45883 (local address 127.15.217.126:42079)
0810 07:15:25.395814 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:25.395816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:25.395827 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:15:25.395902 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:25.395903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:25.395913 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:25.395965 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:25.396297 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:25.396851 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:25.397647 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:25.397874 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:25.397955 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:25.400865 (+ 2910us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.65:45883: BlockingRecv error: recv got EOF from 127.15.217.65:45883 (error 108)
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":97,"threads_started":1}
Aug 10 07:15:25 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:25 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:25.542903 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:25.537820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.66:60923)
0810 07:15:25.537968 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:25.537980 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:25.538055 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:25.538231 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:25.538233 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:25.538425 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:25.538479 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:25.538481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:25.539347 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:25.539348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:25.539910 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:25.539912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:25.540343 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:25.540345 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:25.540414 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:25.540956 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:25.540961 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:25.542800 (+ 1839us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":92,"threads_started":1}
W20260810 07:15:25.542954 26236 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:25.537906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:60923 (local address 127.15.217.126:42079)
0810 07:15:25.537968 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:25.537970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:25.538087 (+ 117us) server_negotiation.cc:407] Connection header received
0810 07:15:25.538126 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:25.538126 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:25.538138 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:25.538218 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:25.538576 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:25.539246 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:25.539999 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:25.540257 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:25.540342 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:25.542886 (+ 2544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.66:60923: BlockingRecv error: recv got EOF from 127.15.217.66:60923 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 10 07:15:25 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:25 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:25.679260 26238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:25.674356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.67:49983)
0810 07:15:25.674518 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:25.674531 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:25.674619 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:25.674764 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:25.674765 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:25.674960 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:25.675013 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:25.675015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:25.675714 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:25.675715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:25.676269 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:25.676271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:25.676725 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:25.676726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:25.676797 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:25.677264 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:25.677269 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:25.679154 (+ 1885us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":88,"threads_started":1}
W20260810 07:15:25.679306 26236 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:25.674450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:49983 (local address 127.15.217.126:42079)
0810 07:15:25.674520 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:25.674522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:25.674597 (+ 75us) server_negotiation.cc:407] Connection header received
0810 07:15:25.674685 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:25.674686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:25.674701 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:25.674749 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:25.675103 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:25.675621 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:25.676365 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:25.676646 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:25.676724 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:25.679240 (+ 2516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.67:49983: BlockingRecv error: recv got EOF from 127.15.217.67:49983 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 10 07:15:26 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:26 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:26.406354 26239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:26.401279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.65:43189)
0810 07:15:26.401453 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:26.401469 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:26.401556 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:26.401705 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:26.401706 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:26.401885 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:26.401944 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:26.401946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:26.402659 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:26.402659 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:26.403274 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:26.403276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:26.403605 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:26.403606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:26.403671 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:26.404152 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:26.404156 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:26.406248 (+ 2092us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":62,"threads_started":1}
W20260810 07:15:26.406415 26240 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:26.401409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:43189 (local address 127.15.217.126:42079)
0810 07:15:26.401569 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:26.401571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:26.401580 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:15:26.401608 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:26.401609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:26.401619 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:26.401680 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:26.402011 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:26.402580 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:26.403341 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:26.403549 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:26.403601 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:26.406338 (+ 2737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.65:43189: BlockingRecv error: recv got EOF from 127.15.217.65:43189 (error 108)
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":77,"threads_started":1}
Aug 10 07:15:26 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:26 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:26.548583 26242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:26.543461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.66:51309)
0810 07:15:26.543644 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:26.543658 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:26.543732 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:26.543867 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:26.543868 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:26.544051 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:26.544106 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:26.544107 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:26.544803 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:26.544804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:26.545378 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:26.545381 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:26.545737 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:26.545737 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:26.545782 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:26.546341 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:26.546347 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:26.548437 (+ 2090us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":78,"threads_started":1}
W20260810 07:15:26.548668 26240 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:26.543623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:51309 (local address 127.15.217.126:42079)
0810 07:15:26.543692 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:26.543694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:26.543715 (+ 21us) server_negotiation.cc:407] Connection header received
0810 07:15:26.543792 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:26.543793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:26.543804 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:26.543846 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:26.544173 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:26.544723 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:26.545460 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:26.545668 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:26.545724 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:26.548588 (+ 2864us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.66:51309: BlockingRecv error: recv got EOF from 127.15.217.66:51309 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 10 07:15:26 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:26 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:26.684687 26243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:26.679873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.67:54117)
0810 07:15:26.680040 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:26.680058 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:26.680129 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:26.680283 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:26.680284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:26.680487 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:26.680542 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:26.680543 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:26.681175 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:26.681176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:26.681829 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:26.681832 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:26.682242 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:26.682243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:26.682312 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:26.682763 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:26.682767 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:26.684591 (+ 1824us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":110,"threads_started":1}
W20260810 07:15:26.684753 26240 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:26.679951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:54117 (local address 127.15.217.126:42079)
0810 07:15:26.680036 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:26.680038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:26.680122 (+ 84us) server_negotiation.cc:407] Connection header received
0810 07:15:26.680202 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:26.680203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:26.680218 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:26.680262 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:26.680596 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:26.681091 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:26.681922 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:26.682152 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:26.682229 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:26.684689 (+ 2460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.67:54117: BlockingRecv error: recv got EOF from 127.15.217.67:54117 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 10 07:15:27 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:27 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:27.411821 26244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:27.406976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.65:48393)
0810 07:15:27.407142 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:27.407156 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:27.407222 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:27.407373 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:27.407374 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:27.407578 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:27.407635 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:27.407637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:27.408324 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:27.408325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:27.408905 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:27.408908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:27.409268 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:27.409269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:27.409315 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:27.409802 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:27.409807 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:27.411756 (+ 1949us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":81,"threads_started":1}
W20260810 07:15:27.411948 26245 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:27.407033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:48393 (local address 127.15.217.126:42079)
0810 07:15:27.407160 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:27.407162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:27.407203 (+ 41us) server_negotiation.cc:407] Connection header received
0810 07:15:27.407277 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:27.407278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:27.407289 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:27.407349 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:27.407713 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:27.408245 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:27.408991 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:27.409197 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:27.409254 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:27.411862 (+ 2608us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.65:48393: BlockingRecv error: recv got EOF from 127.15.217.65:48393 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":56,"threads_started":1}
Aug 10 07:15:27 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:27 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:27.554309 26246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:27.549167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.66:37257)
0810 07:15:27.549406 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:27.549418 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:27.549526 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:27.549710 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:27.549711 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:27.549911 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:27.549968 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:27.549969 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:27.550684 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:27.550686 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:27.551267 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:27.551269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:27.551649 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:27.551650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:27.551707 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:27.552316 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:27.552322 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:27.554244 (+ 1922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":109,"threads_started":1}
W20260810 07:15:27.554466 26245 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:27.549298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:37257 (local address 127.15.217.126:42079)
0810 07:15:27.549398 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:27.549401 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:27.549525 (+ 124us) server_negotiation.cc:407] Connection header received
0810 07:15:27.549613 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:27.549614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:27.549633 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:27.549699 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:27.550083 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:27.550590 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:27.551362 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:27.551572 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:27.551650 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:27.554378 (+ 2728us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.66:37257: BlockingRecv error: recv got EOF from 127.15.217.66:37257 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 10 07:15:27 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:27 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:27.690110 26247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:27.685252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.67:43229)
0810 07:15:27.685420 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:27.685433 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:27.685497 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:27.685708 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:27.685709 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:27.685894 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:27.685949 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:27.685951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:27.686643 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:27.686644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:27.687220 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:27.687221 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:27.687559 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:27.687560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:27.687610 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:27.688088 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:27.688093 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:27.690049 (+ 1956us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":101,"threads_started":1}
W20260810 07:15:27.690270 26245 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:27.685336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:43229 (local address 127.15.217.126:42079)
0810 07:15:27.685386 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:27.685388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:27.685497 (+ 109us) server_negotiation.cc:407] Connection header received
0810 07:15:27.685566 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:27.685567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:27.685581 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:27.685624 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:27.686065 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:27.686564 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:27.687297 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:27.687482 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:27.687535 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:27.690191 (+ 2656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.67:43229: BlockingRecv error: recv got EOF from 127.15.217.67:43229 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 10 07:15:28 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:28 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:28.417615 26249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:28.412453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.65:51427)
0810 07:15:28.412615 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:28.412634 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:28.412690 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:28.412941 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:28.412943 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:28.413127 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:28.413181 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:28.413183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:28.413892 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:28.413892 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:28.414453 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:28.414455 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:28.414896 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:28.414898 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:28.414955 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:28.415550 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:28.415555 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:28.417548 (+ 1993us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":89,"threads_started":1}
W20260810 07:15:28.417867 26250 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:28.412581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:51427 (local address 127.15.217.126:42079)
0810 07:15:28.412711 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:28.412713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:28.412723 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:15:28.412791 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:28.412792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:28.412806 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:28.412893 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:28.413262 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:28.413794 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:28.414589 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:28.414800 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:28.414868 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:28.417775 (+ 2907us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.65:51427: BlockingRecv error: recv got EOF from 127.15.217.65:51427 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":75,"threads_started":1}
Aug 10 07:15:28 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:28 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:28.559861 26251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:28.554840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.66:56815)
0810 07:15:28.555040 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:28.555051 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:28.555116 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:28.555251 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:28.555252 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:28.555428 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:28.555485 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:28.555487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:28.556212 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:28.556213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:28.556820 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:28.556822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:28.557258 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:28.557259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:28.557314 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:28.557890 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:28.557895 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:28.559780 (+ 1885us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":85,"threads_started":1}
W20260810 07:15:28.559937 26250 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:28.554925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:56815 (local address 127.15.217.126:42079)
0810 07:15:28.555053 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:28.555055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:28.555099 (+ 44us) server_negotiation.cc:407] Connection header received
0810 07:15:28.555174 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:28.555175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:28.555186 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:28.555227 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:28.555583 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:28.556131 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:28.556942 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:28.557157 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:28.557226 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:28.559884 (+ 2658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.66:56815: BlockingRecv error: recv got EOF from 127.15.217.66:56815 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 10 07:15:28 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:28 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:28.695731 26252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:28.690664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.67:43231)
0810 07:15:28.690856 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:28.690869 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:28.690922 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:28.691097 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:28.691098 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:28.691279 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:28.691350 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:28.691351 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:28.692063 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:28.692065 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:28.692672 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:28.692675 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:28.693030 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:28.693031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:28.693077 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:28.693567 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:28.693572 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:28.695627 (+ 2055us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":119,"threads_started":1}
W20260810 07:15:28.695784 26250 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:28.690807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:43231 (local address 127.15.217.126:42079)
0810 07:15:28.690890 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:28.690892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:28.690905 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:15:28.690989 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:28.690990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:28.691006 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:28.691068 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:28.691462 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:28.691973 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:28.692756 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:28.692953 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:28.693014 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:28.695715 (+ 2701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.67:43231: BlockingRecv error: recv got EOF from 127.15.217.67:43231 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 10 07:15:29 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:29 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:29.428692 26253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:29.418191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.65:52223)
0810 07:15:29.418375 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:29.418386 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:29.418444 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:29.418596 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:29.418597 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:29.418777 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:29.418837 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:29.418840 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:29.419511 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:29.419511 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:29.420068 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:29.420070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:29.420477 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:29.420478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:29.420538 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:29.426515 (+ 5977us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:29.426521 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:29.428623 (+ 2102us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":108,"threads_started":1}
W20260810 07:15:29.428864 26254 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:29.418314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:52223 (local address 127.15.217.126:42079)
0810 07:15:29.418443 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:29.418445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:29.418453 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:15:29.418496 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:29.418497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:29.418508 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:29.418572 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:29.418905 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:29.419423 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:29.420173 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:29.420363 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:29.420416 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:29.428757 (+ 8341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.65:52223: BlockingRecv error: recv got EOF from 127.15.217.65:52223 (error 108)
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":74,"threads_started":1}
Aug 10 07:15:29 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:29 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:29.565630 26255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:29.560431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.66:54701)
0810 07:15:29.560736 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:29.560747 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:29.560814 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:29.560959 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:29.560960 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:29.561148 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:29.561205 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:29.561208 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:29.562019 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:29.562021 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:29.562580 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:29.562582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:29.562926 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:29.562927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:29.562974 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:29.563558 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:29.563566 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:29.565541 (+ 1975us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":235,"threads_started":1}
W20260810 07:15:29.565717 26254 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:29.560560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:54701 (local address 127.15.217.126:42079)
0810 07:15:29.560674 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:29.560676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:29.560815 (+ 139us) server_negotiation.cc:407] Connection header received
0810 07:15:29.560862 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:29.560863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:29.560876 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:29.560936 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:29.561277 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:29.561917 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:29.562664 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:29.562853 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:29.562905 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:29.565651 (+ 2746us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.66:54701: BlockingRecv error: recv got EOF from 127.15.217.66:54701 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 10 07:15:29 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:29 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:29.702095 26256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:29.696408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.67:51061)
0810 07:15:29.696598 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:29.696612 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:29.696686 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:29.696841 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:29.696842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:29.697084 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:29.697141 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:29.697143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:29.697821 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:29.697822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:29.698452 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:29.698454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:29.699018 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:29.699019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:29.699095 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:29.699676 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:29.699682 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:29.701947 (+ 2265us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":119,"threads_started":1}
W20260810 07:15:29.702150 26254 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:29.696574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:51061 (local address 127.15.217.126:42079)
0810 07:15:29.696627 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:29.696629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:29.696667 (+ 38us) server_negotiation.cc:407] Connection header received
0810 07:15:29.696747 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:29.696748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:29.696761 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:29.696812 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:29.697227 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:29.697734 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:29.698595 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:29.698927 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:29.699006 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:29.702072 (+ 3066us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.67:51061: BlockingRecv error: recv got EOF from 127.15.217.67:51061 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 10 07:15:30 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:30 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:30.434595 26257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:30.429284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.65:50829)
0810 07:15:30.429466 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:30.429481 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:30.429559 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:30.429739 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:30.429740 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:30.429925 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:30.430002 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:30.430005 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:30.430698 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:30.430699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:30.431372 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:30.431376 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:30.431871 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:30.431872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:30.431941 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:30.432476 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:30.432482 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:30.434519 (+ 2037us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":81,"threads_started":1}
W20260810 07:15:30.434764 26259 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:30.429385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:50829 (local address 127.15.217.126:42079)
0810 07:15:30.429498 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:30.429500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:30.429561 (+ 61us) server_negotiation.cc:407] Connection header received
0810 07:15:30.429630 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:30.429630 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:30.429643 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:30.429704 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:30.430077 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:30.430614 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:30.431454 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:30.431785 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:30.431857 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:30.434659 (+ 2802us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.65:50829: BlockingRecv error: recv got EOF from 127.15.217.65:50829 (error 108)
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":53,"threads_started":1}
Aug 10 07:15:30 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:30 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:30.570976 26260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:30.566169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.66:40093)
0810 07:15:30.566325 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:30.566340 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:30.566402 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:30.566552 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:30.566553 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:30.566733 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:30.566789 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:30.566790 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:30.567454 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:30.567455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:30.567998 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:30.567999 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:30.568358 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:30.568359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:30.568401 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:30.568980 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:30.568985 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:30.570902 (+ 1917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":93,"threads_started":1}
W20260810 07:15:30.571080 26259 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:30.566262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:40093 (local address 127.15.217.126:42079)
0810 07:15:30.566313 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:30.566315 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:30.566388 (+ 73us) server_negotiation.cc:407] Connection header received
0810 07:15:30.566451 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:30.566452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:30.566463 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:30.566524 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:30.566868 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:30.567380 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:30.568080 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:30.568288 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:30.568344 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:30.571008 (+ 2664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.66:40093: BlockingRecv error: recv got EOF from 127.15.217.66:40093 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 10 07:15:30 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:30 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:30.707965 26261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:30.702802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.67:33139)
0810 07:15:30.702936 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:30.702948 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:30.703013 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:30.703140 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:30.703140 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:30.703347 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:30.703403 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:30.703404 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:30.704155 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:30.704156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:30.704760 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:30.704762 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:30.705228 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:30.705229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:30.705271 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:30.705846 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:30.705852 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:30.707904 (+ 2052us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":74,"threads_started":1}
W20260810 07:15:30.708127 26259 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:30.702856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:33139 (local address 127.15.217.126:42079)
0810 07:15:30.702910 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:30.702913 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:30.703020 (+ 107us) server_negotiation.cc:407] Connection header received
0810 07:15:30.703070 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:30.703070 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:30.703084 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:30.703133 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:30.703528 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:30.704078 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:30.704917 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:30.705150 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:30.705217 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:30.708033 (+ 2816us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.67:33139: BlockingRecv error: recv got EOF from 127.15.217.67:33139 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 10 07:15:31 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:31 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:31.440250 26262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.435104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.65:50819)
0810 07:15:31.435277 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:31.435289 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:31.435353 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:31.435498 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:31.435499 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:31.435684 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:31.435739 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.435740 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.436566 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.436567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.437113 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.437115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.437522 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.437523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.437600 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.438115 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:31.438119 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:31.440189 (+ 2070us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":104,"threads_started":1}
W20260810 07:15:31.440409 26263 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:31.435196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:50819 (local address 127.15.217.126:42079)
0810 07:15:31.435333 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:31.435335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:31.435346 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:15:31.435406 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:31.435407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:31.435418 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:31.435481 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:31.435836 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.436503 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.437191 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.437436 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.437521 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.440317 (+ 2796us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.65:50819: BlockingRecv error: recv got EOF from 127.15.217.65:50819 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":76,"threads_started":1}
Aug 10 07:15:31 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:31 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:31.576435 26264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.571591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.66:46225)
0810 07:15:31.571780 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:31.571792 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:31.571860 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:31.572013 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:31.572013 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:31.572198 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:31.572252 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.572254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.573008 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.573009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.573555 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.573557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.573952 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.573953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.573996 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.574439 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:31.574443 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:31.576372 (+ 1929us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":99,"threads_started":1}
W20260810 07:15:31.576633 26263 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:31.571652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:46225 (local address 127.15.217.126:42079)
0810 07:15:31.571729 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:31.571732 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:31.571884 (+ 152us) server_negotiation.cc:407] Connection header received
0810 07:15:31.571922 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:31.571924 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:31.571937 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:31.571998 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:31.572319 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.572936 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.573630 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.573866 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.573953 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.576547 (+ 2594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.66:46225: BlockingRecv error: recv got EOF from 127.15.217.66:46225 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 10 07:15:31 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:31 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:31.713303 26265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.708516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.67:37901)
0810 07:15:31.708691 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:31.708702 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:31.708755 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:31.708992 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:31.708993 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:31.709181 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:31.709234 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.709236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.709908 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.709909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.710455 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.710456 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.710812 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.710813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.710865 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.711342 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:31.711346 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:31.713230 (+ 1884us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":65,"threads_started":1}
W20260810 07:15:31.713407 26263 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:31.708697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:37901 (local address 127.15.217.126:42079)
0810 07:15:31.708863 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:31.708865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:31.708876 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:15:31.708904 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:31.708904 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:31.708917 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:31.708967 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:31.709306 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.709828 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.710536 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.710735 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.710790 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.713335 (+ 2545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.67:37901: BlockingRecv error: recv got EOF from 127.15.217.67:37901 (error 108)
Metrics: {"server-negotiator.queue_time_us":141}
Aug 10 07:15:32 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:32 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:32.445669 26267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:32.440771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.65:34785)
0810 07:15:32.440918 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:32.440958 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:32.441012 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:32.441157 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:32.441158 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:32.441347 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:32.441401 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:32.441402 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.442087 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.442088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:32.442657 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:32.442659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.443029 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.443030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:32.443083 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.443571 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:32.443576 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:32.445594 (+ 2018us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":90,"threads_started":1}
W20260810 07:15:32.445832 26268 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:32.440830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:34785 (local address 127.15.217.126:42079)
0810 07:15:32.441015 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:32.441017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:32.441029 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:15:32.441056 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:32.441057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:32.441070 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:32.441140 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:32.441476 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.441989 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.442741 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.442942 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.442999 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.445722 (+ 2723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.65:34785: BlockingRecv error: recv got EOF from 127.15.217.65:34785 (error 108)
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":123,"threads_started":1}
Aug 10 07:15:32 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:32 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:32.582139 26269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:32.576980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.66:59431)
0810 07:15:32.577181 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:32.577195 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:32.577287 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:32.577434 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:32.577435 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:32.577652 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:32.577704 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:32.577706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.578440 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.578442 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:32.579077 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:32.579080 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.579499 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.579500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:32.579566 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.580014 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:32.580019 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:32.582012 (+ 1993us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":101,"threads_started":1}
W20260810 07:15:32.582190 26268 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:32.577115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:59431 (local address 127.15.217.126:42079)
0810 07:15:32.577217 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:32.577220 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:32.577287 (+ 67us) server_negotiation.cc:407] Connection header received
0810 07:15:32.577366 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:32.577367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:32.577379 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:32.577426 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:32.577813 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.578349 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.579163 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.579377 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.579497 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.582116 (+ 2619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.66:59431: BlockingRecv error: recv got EOF from 127.15.217.66:59431 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 10 07:15:32 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:32 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:32.719084 26270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:32.713885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.67:43887)
0810 07:15:32.714059 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:32.714074 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:32.714185 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:32.714371 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:32.714372 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:32.714581 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:32.714640 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:32.714642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.715406 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.715408 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:32.715994 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:32.715995 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.716418 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.716419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:32.716551 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.717042 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:32.717047 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:32.718986 (+ 1939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":94,"threads_started":1}
W20260810 07:15:32.719156 26268 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:32.714013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:43887 (local address 127.15.217.126:42079)
0810 07:15:32.714094 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:32.714097 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:32.714162 (+ 65us) server_negotiation.cc:407] Connection header received
0810 07:15:32.714265 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:32.714265 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:32.714281 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:32.714362 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:32.714727 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.715313 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.716095 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.716327 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.716410 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.719105 (+ 2695us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.67:43887: BlockingRecv error: recv got EOF from 127.15.217.67:43887 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 10 07:15:33 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:33 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:33.451596 26271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:33.446262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.65:43265)
0810 07:15:33.446420 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:33.446431 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:33.446486 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:33.446643 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:33.446644 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:33.446857 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:33.446931 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:33.446933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:33.447950 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:33.447951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:33.448612 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:33.448615 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:33.448938 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:33.448939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:33.448985 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:33.449525 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:33.449538 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:33.451511 (+ 1973us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":87,"threads_started":1}
W20260810 07:15:33.451704 26273 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:33.446349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:43265 (local address 127.15.217.126:42079)
0810 07:15:33.446483 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:33.446485 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:33.446494 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:15:33.446550 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:33.446551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:33.446563 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:33.446619 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:33.447070 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:33.447865 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:33.448681 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:33.448873 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:33.448926 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:33.451622 (+ 2696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.65:43265: BlockingRecv error: recv got EOF from 127.15.217.65:43265 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":78,"threads_started":1}
Aug 10 07:15:33 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:33 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:33.587702 26274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:33.582677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.66:40805)
0810 07:15:33.582818 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:33.582830 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:33.582898 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:33.583014 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:33.583015 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:33.583196 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:33.583257 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:33.583259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:33.583971 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:33.583972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:33.584682 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:33.584685 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:33.585102 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:33.585103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:33.585154 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:33.585633 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:33.585639 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:33.587586 (+ 1947us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":83,"threads_started":1}
W20260810 07:15:33.587740 26273 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:33.582789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:40805 (local address 127.15.217.126:42079)
0810 07:15:33.582840 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:33.582842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:33.582896 (+ 54us) server_negotiation.cc:407] Connection header received
0810 07:15:33.582947 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:33.582947 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:33.582958 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:33.583003 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:33.583367 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:33.583884 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:33.584768 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:33.584977 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:33.585045 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:33.587671 (+ 2626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.66:40805: BlockingRecv error: recv got EOF from 127.15.217.66:40805 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 10 07:15:33 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:33 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:33.725222 26276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:33.719747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.67:49593)
0810 07:15:33.719943 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:33.719955 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:33.720035 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:33.720212 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:33.720214 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:33.720416 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:33.720525 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:33.720527 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:33.721523 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:33.721525 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:33.722069 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:33.722070 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:33.722454 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:33.722455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:33.722516 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:33.722996 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:33.723002 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:33.725116 (+ 2114us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":70,"threads_started":1}
W20260810 07:15:33.725277 26273 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:33.719856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:49593 (local address 127.15.217.126:42079)
0810 07:15:33.719924 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:33.719927 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:33.720012 (+ 85us) server_negotiation.cc:407] Connection header received
0810 07:15:33.720105 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:33.720106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:33.720121 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:33.720188 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:33.720660 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:33.721410 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:33.722157 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:33.722379 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:33.722441 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:33.725208 (+ 2767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.67:49593: BlockingRecv error: recv got EOF from 127.15.217.67:49593 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 10 07:15:34 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:34 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:34.463869 26277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:34.452233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.65:46643)
0810 07:15:34.452422 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:34.452438 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:34.452560 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:34.452730 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:34.452731 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:34.452933 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:34.452990 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:34.452992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:34.453682 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:34.453683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:34.454251 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:34.454252 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:34.454689 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:34.454690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:34.454762 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:34.461797 (+ 7035us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:34.461804 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:34.463803 (+ 1999us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":81,"threads_started":1}
W20260810 07:15:34.464026 26278 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:34.452377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:46643 (local address 127.15.217.126:42079)
0810 07:15:34.452560 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:34.452563 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:34.452574 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:15:34.452608 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:34.452608 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:34.452620 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:34.452684 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:34.453083 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:34.453599 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:34.454376 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:34.454593 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:34.454686 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:34.463933 (+ 9247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.65:46643: BlockingRecv error: recv got EOF from 127.15.217.65:46643 (error 108)
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":102,"threads_started":1}
Aug 10 07:15:34 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:34 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:34.593230 26279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:34.588295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.66:42071)
0810 07:15:34.588510 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:34.588526 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:34.588605 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:34.588757 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:34.588758 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:34.589000 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:34.589073 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:34.589075 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:34.589725 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:34.589726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:34.590364 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:34.590366 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:34.590745 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:34.590746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:34.590801 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:34.591278 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:34.591283 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:34.593167 (+ 1884us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":77,"threads_started":1}
W20260810 07:15:34.593346 26278 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:34.588515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:42071 (local address 127.15.217.126:42079)
0810 07:15:34.588569 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:34.588571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:34.588586 (+ 15us) server_negotiation.cc:407] Connection header received
0810 07:15:34.588668 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:34.588669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:34.588680 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:34.588728 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:34.589147 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:34.589644 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:34.590449 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:34.590652 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:34.590709 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:34.593268 (+ 2559us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.66:42071: BlockingRecv error: recv got EOF from 127.15.217.66:42071 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 10 07:15:34 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:34 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:34.731330 26280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:34.725816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.67:40189)
0810 07:15:34.725940 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:34.725954 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:34.726119 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:34.726270 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:34.726271 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:34.726530 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:34.726593 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:34.726595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:34.727347 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:34.727348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:34.728226 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:34.728230 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:34.728785 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:34.728786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:34.728848 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:34.729380 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:34.729385 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:34.731256 (+ 1871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":66,"threads_started":1}
W20260810 07:15:34.731498 26278 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:34.725884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:40189 (local address 127.15.217.126:42079)
0810 07:15:34.725939 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:34.725942 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:34.726114 (+ 172us) server_negotiation.cc:407] Connection header received
0810 07:15:34.726184 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:34.726185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:34.726201 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:34.726261 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:34.726682 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:34.727244 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:34.728359 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:34.728680 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:34.728764 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:34.731412 (+ 2648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.67:40189: BlockingRecv error: recv got EOF from 127.15.217.67:40189 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 10 07:15:35 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:35 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:35.469924 26281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:35.464438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.65:60823)
0810 07:15:35.464629 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:35.464641 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:35.464706 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:35.464886 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:35.464886 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:35.465082 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:35.465138 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:35.465140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:35.465786 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:35.465787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:35.466339 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:35.466340 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:35.466742 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:35.466743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:35.466823 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:35.467487 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:35.467493 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:35.469848 (+ 2355us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":131,"threads_started":1}
W20260810 07:15:35.470080 26282 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:35.464610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:60823 (local address 127.15.217.126:42079)
0810 07:15:35.464749 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:35.464750 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:35.464759 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:15:35.464786 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:35.464787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:35.464798 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:35.464867 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:35.465206 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:35.465715 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:35.466415 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:35.466612 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:35.466726 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:35.469969 (+ 3243us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.65:60823: BlockingRecv error: recv got EOF from 127.15.217.65:60823 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":75,"threads_started":1}
Aug 10 07:15:35 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:35 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:35.599426 26283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:35.593851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.66:32829)
0810 07:15:35.594065 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:35.594080 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:35.594194 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:35.594398 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:35.594399 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:35.594618 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:35.594673 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:35.594676 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:35.595451 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:35.595453 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:35.596049 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:35.596051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:35.596527 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:35.596529 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:35.596597 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:35.597157 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:35.597163 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:35.599314 (+ 2151us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":100,"threads_started":1}
W20260810 07:15:35.599469 26282 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:35.593974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:32829 (local address 127.15.217.126:42079)
0810 07:15:35.594061 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:35.594064 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:35.594171 (+ 107us) server_negotiation.cc:407] Connection header received
0810 07:15:35.594298 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:35.594299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:35.594320 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:35.594387 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:35.594774 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:35.595342 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:35.596178 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:35.596407 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:35.596527 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:35.599400 (+ 2873us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.66:32829: BlockingRecv error: recv got EOF from 127.15.217.66:32829 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 10 07:15:35 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:35 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.67@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:35.736879 26284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:35.731895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.67:37217)
0810 07:15:35.732053 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:35.732089 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:35.732152 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:35.732314 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:35.732315 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:35.732575 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:35.732641 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:35.732643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:35.733308 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:35.733308 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:35.733830 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:35.733831 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:35.734170 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:35.734171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:35.734220 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:35.734766 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:35.734771 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:35.736815 (+ 2044us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":86,"threads_started":1}
W20260810 07:15:35.736985 26282 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:35.731989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:37217 (local address 127.15.217.126:42079)
0810 07:15:35.732083 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:35.732086 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:35.732138 (+ 52us) server_negotiation.cc:407] Connection header received
0810 07:15:35.732217 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:35.732218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:35.732234 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:35.732292 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:35.732711 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:35.733237 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:35.733901 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:35.734100 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:35.734151 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:35.736913 (+ 2762us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.67:37217: BlockingRecv error: recv got EOF from 127.15.217.67:37217 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 10 07:15:36 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:36 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.65@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:36.475889 26285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:36.470744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.65:35293)
0810 07:15:36.470903 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:36.470913 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:36.470966 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:36.471142 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:36.471143 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:36.471314 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:36.471370 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:36.471372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:36.472181 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:36.472182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:36.472788 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:36.472790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:36.473171 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:36.473173 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:36.473218 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:36.473778 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:36.473786 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:36.475827 (+ 2041us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":97,"threads_started":1}
W20260810 07:15:36.476032 26286 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:36.470827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:35293 (local address 127.15.217.126:42079)
0810 07:15:36.470975 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:36.470982 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:36.471017 (+ 35us) server_negotiation.cc:407] Connection header received
0810 07:15:36.471045 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:36.471045 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:36.471058 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:36.471121 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:36.471470 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:36.472035 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:36.472862 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:36.473096 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:36.473157 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:36.475939 (+ 2782us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.65:35293: BlockingRecv error: recv got EOF from 127.15.217.65:35293 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":89,"threads_started":1}
Aug 10 07:15:36 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:36 dist-test-slave-10pc krb5kdc[25150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.217.66@KRBTEST.COM for oryx/127.15.217.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:36.605342 26287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:36.600104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42079 (local address 127.15.217.66:40775)
0810 07:15:36.600274 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:36.600293 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:36.600354 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:36.600528 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:36.600530 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:36.600709 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:36.600764 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:36.600766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:36.601432 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:36.601434 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:36.601996 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:36.601998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:36.602419 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:36.602420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:36.602479 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:36.603037 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:36.603043 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:36.605239 (+ 2196us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.217.126:42079: Server oryx/127.15.217.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":109,"threads_started":1}
W20260810 07:15:36.605465 26286 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:36.600179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:40775 (local address 127.15.217.126:42079)
0810 07:15:36.600276 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:36.600278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:36.600342 (+ 64us) server_negotiation.cc:407] Connection header received
0810 07:15:36.600406 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:36.600407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:36.600417 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:36.600516 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:36.600833 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:36.601342 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:36.602114 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:36.602335 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:36.602399 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:36.605389 (+ 2990us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.217.66:40775: BlockingRecv error: recv got EOF from 127.15.217.66:40775 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 07:15:36.650498 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 25772
I20260810 07:15:36.659103 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 25934
I20260810 07:15:36.666882 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 26076
I20260810 07:15:36.676858 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 25699
2026-08-10T07:15:36Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17654 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-10T07:15:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:15:36Z Disabled control of system clock
I20260810 07:15:36.709008 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:35287
--webserver_interface=127.15.217.126
--webserver_port=0
--builtin_ntp_servers=127.15.217.84:35377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:35287
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 07:15:36.795552 26291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:36.795776 26291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:36.795799 26291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:36.797056 26291 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:36.797103 26291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:36.797117 26291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:36.797130 26291 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:36.797143 26291 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:15:36.798557 26291 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:35377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:35287
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:35287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:36.798831 26291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:36.799162 26291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:36.804669 26301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:36.804864 26291 server_base.cc:1034] running on GCE node
W20260810 07:15:36.804867 26299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:36.804878 26297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:36.805433 26291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:36.805771 26291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:36.806936 26291 hybrid_clock.cc:648] HybridClock initialized: now 1786346136806908 us; error 36 us; skew 500 ppm
I20260810 07:15:36.808218 26291 webserver.cc:460] Webserver started at http://127.15.217.126:37843/ using document root <none> and password file <none>
I20260810 07:15:36.808522 26291 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:36.808602 26291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:36.808751 26291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:36.809877 26291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "78cd3c336b984a0892b350f437fc9b59"
format_stamp: "Formatted at 2026-08-10 07:15:36 on dist-test-slave-10pc"
server_key: "8f9706e29658411192586fcc9776ae16"
server_key_iv: "8bb42708e99eb1776bf8c17d2c5e66cc"
server_key_version: "encryptionkey@0"
I20260810 07:15:36.810278 26291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "78cd3c336b984a0892b350f437fc9b59"
format_stamp: "Formatted at 2026-08-10 07:15:36 on dist-test-slave-10pc"
server_key: "8f9706e29658411192586fcc9776ae16"
server_key_iv: "8bb42708e99eb1776bf8c17d2c5e66cc"
server_key_version: "encryptionkey@0"
I20260810 07:15:36.811620 26291 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:15:36.812379 26307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:36.812559 26291 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 07:15:36.812669 26291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "78cd3c336b984a0892b350f437fc9b59"
format_stamp: "Formatted at 2026-08-10 07:15:36 on dist-test-slave-10pc"
server_key: "8f9706e29658411192586fcc9776ae16"
server_key_iv: "8bb42708e99eb1776bf8c17d2c5e66cc"
server_key_version: "encryptionkey@0"
I20260810 07:15:36.812750 26291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:36.835852 26291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:36.836262 26291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:36.836436 26291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:36.840658 26291 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:35287
I20260810 07:15:36.840698 26364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:35287 every 8 connection(s)
I20260810 07:15:36.841104 26291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:15:36.841864 26365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:36.844276 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 26291
I20260810 07:15:36.844388 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:15:36.844655 16229 external_mini_cluster.cc:1428] Setting key a5bd2cc8bc726b3bb87245e6bd5c843c
I20260810 07:15:36.844990 26365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78cd3c336b984a0892b350f437fc9b59: Bootstrap starting.
I20260810 07:15:36.845794 26365 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78cd3c336b984a0892b350f437fc9b59: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:36.846159 26365 log.cc:824] T 00000000000000000000000000000000 P 78cd3c336b984a0892b350f437fc9b59: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:36.847023 26365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78cd3c336b984a0892b350f437fc9b59: No bootstrap required, opened a new log
I20260810 07:15:36.849588 26365 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78cd3c336b984a0892b350f437fc9b59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78cd3c336b984a0892b350f437fc9b59" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35287 } }
I20260810 07:15:36.849778 26365 raft_consensus.cc:374] T 00000000000000000000000000000000 P 78cd3c336b984a0892b350f437fc9b59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:36.849823 26365 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78cd3c336b984a0892b350f437fc9b59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78cd3c336b984a0892b350f437fc9b59, State: Initialized, Role: FOLLOWER
I20260810 07:15:36.850004 26365 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78cd3c336b984a0892b350f437fc9b59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78cd3c336b984a0892b350f437fc9b59" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35287 } }
I20260810 07:15:36.850119 26365 raft_consensus.cc:388] T 00000000000000000000000000000000 P 78cd3c336b984a0892b350f437fc9b59 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:15:36.850167 26365 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78cd3c336b984a0892b350f437fc9b59 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:15:36.850214 26365 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78cd3c336b984a0892b350f437fc9b59 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:36.851073 26365 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78cd3c336b984a0892b350f437fc9b59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78cd3c336b984a0892b350f437fc9b59" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35287 } }
I20260810 07:15:36.851193 26365 leader_election.cc:302] T 00000000000000000000000000000000 P 78cd3c336b984a0892b350f437fc9b59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 78cd3c336b984a0892b350f437fc9b59; no voters:
I20260810 07:15:36.851359 26365 leader_election.cc:288] T 00000000000000000000000000000000 P 78cd3c336b984a0892b350f437fc9b59 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:15:36.851440 26370 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78cd3c336b984a0892b350f437fc9b59 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:36.851534 26368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:36.845305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59116 (local address 127.15.217.126:35287)
0810 07:15:36.845566 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:36.845575 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:36.845611 (+ 36us) server_negotiation.cc:407] Connection header received
0810 07:15:36.846248 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:36.846258 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:36.846350 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:36.846497 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:36.846774 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:36.848206 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:36.849066 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:36.849770 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:36.849920 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:36.850716 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:36.850721 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:36.850722 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:36.850946 (+ 224us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:36.850994 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:36.851011 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:36.851087 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:36.851169 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 07:15:36.851261 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":105,"threads_started":1}
I20260810 07:15:36.851641 26370 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78cd3c336b984a0892b350f437fc9b59 [term 1 LEADER]: Becoming Leader. State: Replica: 78cd3c336b984a0892b350f437fc9b59, State: Running, Role: LEADER
I20260810 07:15:36.851668 26365 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78cd3c336b984a0892b350f437fc9b59 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:36.851770 26370 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78cd3c336b984a0892b350f437fc9b59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78cd3c336b984a0892b350f437fc9b59" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35287 } }
I20260810 07:15:36.852080 26370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78cd3c336b984a0892b350f437fc9b59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "78cd3c336b984a0892b350f437fc9b59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78cd3c336b984a0892b350f437fc9b59" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35287 } } }
I20260810 07:15:36.852155 26370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78cd3c336b984a0892b350f437fc9b59 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:36.852303 26370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78cd3c336b984a0892b350f437fc9b59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78cd3c336b984a0892b350f437fc9b59. Latest consensus state: current_term: 1 leader_uuid: "78cd3c336b984a0892b350f437fc9b59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78cd3c336b984a0892b350f437fc9b59" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35287 } } }
I20260810 07:15:36.852418 26370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78cd3c336b984a0892b350f437fc9b59 [sys.catalog]: This master's current role is: LEADER
W20260810 07:15:36.853751 26386 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 78cd3c336b984a0892b350f437fc9b59: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:15:36.853832 26386 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:15:36.853888 26375 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:36.854056 26375 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:15:36.855727 26375 catalog_manager.cc:1347] Generated new cluster ID: c59576ae4acf4d90a45295773cbf9429
I20260810 07:15:36.855788 26375 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:15:36.873531 26375 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:15:36.874329 26375 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:36.879732 26375 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 78cd3c336b984a0892b350f437fc9b59: Generated new TSK 0
I20260810 07:15:36.879961 26375 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:15:36.886674 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.65:0
--local_ip_for_outbound_sockets=127.15.217.65
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:35287
--builtin_ntp_servers=127.15.217.84:35377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260810 07:15:36.973309 26389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:36.973558 26389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:36.973598 26389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:36.974828 26389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:36.974897 26389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:36.974941 26389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.65
I20260810 07:15:36.976351 26389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:35377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:35287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.65
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:36.976657 26389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:36.976987 26389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:36.982375 26397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:36.982375 26394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:36.982365 26395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:36.982676 26389 server_base.cc:1034] running on GCE node
I20260810 07:15:36.982884 26389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:36.983156 26389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:36.984326 26389 hybrid_clock.cc:648] HybridClock initialized: now 1786346136984321 us; error 40 us; skew 500 ppm
I20260810 07:15:36.985805 26389 webserver.cc:460] Webserver started at http://127.15.217.65:43291/ using document root <none> and password file <none>
I20260810 07:15:36.986085 26389 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:36.986162 26389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:36.986321 26389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:36.987447 26389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/data/instance:
uuid: "1770fee48c974c6e9dc2d500190ce99a"
format_stamp: "Formatted at 2026-08-10 07:15:36 on dist-test-slave-10pc"
server_key: "2fb5c6b9b64700673575d297a3f5fad1"
server_key_iv: "93187bc80a945a1ec0066a6dacb0a152"
server_key_version: "encryptionkey@0"
I20260810 07:15:36.987830 26389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance:
uuid: "1770fee48c974c6e9dc2d500190ce99a"
format_stamp: "Formatted at 2026-08-10 07:15:36 on dist-test-slave-10pc"
server_key: "2fb5c6b9b64700673575d297a3f5fad1"
server_key_iv: "93187bc80a945a1ec0066a6dacb0a152"
server_key_version: "encryptionkey@0"
I20260810 07:15:36.989307 26389 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:15:36.990147 26404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:36.990374 26389 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:36.990485 26389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/wal
uuid: "1770fee48c974c6e9dc2d500190ce99a"
format_stamp: "Formatted at 2026-08-10 07:15:36 on dist-test-slave-10pc"
server_key: "2fb5c6b9b64700673575d297a3f5fad1"
server_key_iv: "93187bc80a945a1ec0066a6dacb0a152"
server_key_version: "encryptionkey@0"
I20260810 07:15:36.990563 26389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:37.001111 26389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:37.001436 26389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:37.001597 26389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:37.001823 26389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:37.002195 26389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:37.002254 26389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:37.002296 26389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:37.002324 26389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:37.008285 26389 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.65:40189
I20260810 07:15:37.008373 26529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.65:40189 every 8 connection(s)
I20260810 07:15:37.008805 26389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
I20260810 07:15:37.013196 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 26389
I20260810 07:15:37.013321 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance
I20260810 07:15:37.013571 16229 external_mini_cluster.cc:1428] Setting key 059fec939c6d2a4d1f5ff8bd89dfd0fb
I20260810 07:15:37.014917 26368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:37.009708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:39765 (local address 127.15.217.126:35287)
0810 07:15:37.009776 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:37.009778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:37.010084 (+ 306us) server_negotiation.cc:407] Connection header received
0810 07:15:37.010610 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:37.010611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:37.010628 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:37.010693 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:37.011405 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:37.011961 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:37.013116 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:37.013464 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:37.013580 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:37.014602 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:37.014603 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:37.014604 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:37.014609 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:37.014629 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:37.014634 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:37.014704 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:37.014773 (+ 69us) server_negotiation.cc:299] Negotiation successful
0810 07:15:37.014802 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 07:15:37.014910 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.66:0
--local_ip_for_outbound_sockets=127.15.217.66
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:35287
--builtin_ntp_servers=127.15.217.84:35377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 07:15:37.015094 26532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:37.009707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35287 (local address 127.15.217.65:39765)
0810 07:15:37.009931 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:37.009966 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:37.010489 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:37.010774 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:37.010777 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:37.010787 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:37.011300 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:37.011304 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:37.012110 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:37.012111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:37.013016 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:37.013019 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:37.013595 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:37.013596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:37.013764 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:37.014459 (+ 695us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:15:37.014459 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:37.014492 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:15:37.014516 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:37.014709 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:37.014715 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:37.014717 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 07:15:37.014770 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 07:15:37.014863 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":113,"threads_started":1}
I20260810 07:15:37.017186 26530 heartbeater.cc:347] Connected to a master server at 127.15.217.126:35287
I20260810 07:15:37.017306 26530 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:37.017555 26530 heartbeater.cc:511] Master 127.15.217.126:35287 requested a full tablet report, sending...
I20260810 07:15:37.018177 26326 ts_manager.cc:194] Registered new tserver with Master: 1770fee48c974c6e9dc2d500190ce99a (127.15.217.65:40189)
I20260810 07:15:37.018914 26326 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.217.65:39765
W20260810 07:15:37.101640 26533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:37.101900 26533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:37.101940 26533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:37.103132 26533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:37.103201 26533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:37.103243 26533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.66
I20260810 07:15:37.104661 26533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:35377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:35287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.66
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:37.104974 26533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:37.105317 26533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:15:37.110829 26533 server_base.cc:1034] running on GCE node
W20260810 07:15:37.110797 26539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:37.110809 26542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:37.110874 26540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:37.111229 26533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:37.111536 26533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:37.112741 26533 hybrid_clock.cc:648] HybridClock initialized: now 1786346137112734 us; error 37 us; skew 500 ppm
I20260810 07:15:37.114302 26533 webserver.cc:460] Webserver started at http://127.15.217.66:41441/ using document root <none> and password file <none>
I20260810 07:15:37.114581 26533 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:37.114658 26533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:37.114816 26533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:37.116096 26533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/data/instance:
uuid: "346d341e43414975b8174cfa866fc958"
format_stamp: "Formatted at 2026-08-10 07:15:37 on dist-test-slave-10pc"
server_key: "dfa57adc20b6734c624169a5644e92f0"
server_key_iv: "ba5df974add6f0a1fb947c085cdd731e"
server_key_version: "encryptionkey@0"
I20260810 07:15:37.116561 26533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance:
uuid: "346d341e43414975b8174cfa866fc958"
format_stamp: "Formatted at 2026-08-10 07:15:37 on dist-test-slave-10pc"
server_key: "dfa57adc20b6734c624169a5644e92f0"
server_key_iv: "ba5df974add6f0a1fb947c085cdd731e"
server_key_version: "encryptionkey@0"
I20260810 07:15:37.118040 26533 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:37.118938 26549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:37.119172 26533 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:37.119278 26533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/wal
uuid: "346d341e43414975b8174cfa866fc958"
format_stamp: "Formatted at 2026-08-10 07:15:37 on dist-test-slave-10pc"
server_key: "dfa57adc20b6734c624169a5644e92f0"
server_key_iv: "ba5df974add6f0a1fb947c085cdd731e"
server_key_version: "encryptionkey@0"
I20260810 07:15:37.119369 26533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:37.131593 26533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:37.131927 26533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:37.132089 26533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:37.132339 26533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:37.132769 26533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:37.132827 26533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:37.132861 26533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:37.132892 26533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:37.138953 26533 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.66:45051
I20260810 07:15:37.139036 26670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.66:45051 every 8 connection(s)
I20260810 07:15:37.139410 26533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
I20260810 07:15:37.141323 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 26533
I20260810 07:15:37.141407 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance
I20260810 07:15:37.141623 16229 external_mini_cluster.cc:1428] Setting key f58f50f60a9c5966486b438f4e64b8da
I20260810 07:15:37.142751 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.67:0
--local_ip_for_outbound_sockets=127.15.217.67
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:35287
--builtin_ntp_servers=127.15.217.84:35377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 07:15:37.145828 26368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:37.140194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:40503 (local address 127.15.217.126:35287)
0810 07:15:37.140279 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:37.140296 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:37.140549 (+ 253us) server_negotiation.cc:407] Connection header received
0810 07:15:37.141144 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:37.141145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:37.141158 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:37.141209 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:37.142179 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:37.142930 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:37.144083 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:37.144401 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:37.144518 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:37.145555 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:37.145557 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:37.145557 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:37.145561 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:37.145578 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:37.145582 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:37.145627 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:37.145741 (+ 114us) server_negotiation.cc:299] Negotiation successful
0810 07:15:37.145764 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 07:15:37.146135 26673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:37.140184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35287 (local address 127.15.217.66:40503)
0810 07:15:37.140395 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:37.140431 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:37.141019 (+ 588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:37.141285 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:37.141289 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:37.141299 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:37.142080 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:37.142088 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:37.143046 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:37.143047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:37.144001 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:37.144004 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:37.144590 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:37.144591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:37.144771 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:37.145373 (+ 602us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:15:37.145374 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:37.145416 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:15:37.145448 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:37.145652 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:37.145659 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:37.145662 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:15:37.145736 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 07:15:37.145865 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":75,"threads_started":1}
I20260810 07:15:37.146699 26671 heartbeater.cc:347] Connected to a master server at 127.15.217.126:35287
I20260810 07:15:37.146807 26671 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:37.147018 26671 heartbeater.cc:511] Master 127.15.217.126:35287 requested a full tablet report, sending...
I20260810 07:15:37.147434 26326 ts_manager.cc:194] Registered new tserver with Master: 346d341e43414975b8174cfa866fc958 (127.15.217.66:45051)
I20260810 07:15:37.147850 26326 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.217.66:40503
W20260810 07:15:37.231305 26674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:37.231521 26674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:37.231544 26674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:37.233353 26674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:37.233403 26674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:37.233431 26674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.67
I20260810 07:15:37.234830 26674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:35377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:35287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.67
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:37.235106 26674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:37.235425 26674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:37.240603 26683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:37.240684 26680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:37.240603 26681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:37.240935 26674 server_base.cc:1034] running on GCE node
I20260810 07:15:37.241175 26674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:37.241460 26674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:37.242633 26674 hybrid_clock.cc:648] HybridClock initialized: now 1786346137242609 us; error 35 us; skew 500 ppm
I20260810 07:15:37.244094 26674 webserver.cc:460] Webserver started at http://127.15.217.67:38383/ using document root <none> and password file <none>
I20260810 07:15:37.244364 26674 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:37.244436 26674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:37.244625 26674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:37.245766 26674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/data/instance:
uuid: "926eda0e3a7a4f7496980f84ef3d815b"
format_stamp: "Formatted at 2026-08-10 07:15:37 on dist-test-slave-10pc"
server_key: "59038e42926ae5d1eef432baee808638"
server_key_iv: "2aeac7f113ccaee64efde4b7af27ce36"
server_key_version: "encryptionkey@0"
I20260810 07:15:37.246155 26674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance:
uuid: "926eda0e3a7a4f7496980f84ef3d815b"
format_stamp: "Formatted at 2026-08-10 07:15:37 on dist-test-slave-10pc"
server_key: "59038e42926ae5d1eef432baee808638"
server_key_iv: "2aeac7f113ccaee64efde4b7af27ce36"
server_key_version: "encryptionkey@0"
I20260810 07:15:37.247507 26674 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:15:37.248291 26689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:37.248526 26674 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:37.248641 26674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/wal
uuid: "926eda0e3a7a4f7496980f84ef3d815b"
format_stamp: "Formatted at 2026-08-10 07:15:37 on dist-test-slave-10pc"
server_key: "59038e42926ae5d1eef432baee808638"
server_key_iv: "2aeac7f113ccaee64efde4b7af27ce36"
server_key_version: "encryptionkey@0"
I20260810 07:15:37.248733 26674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:37.265311 26674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:37.265697 26674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:37.265872 26674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:37.266137 26674 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:37.266518 26674 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:37.266577 26674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:37.266623 26674 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:37.266647 26674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:37.272676 26674 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.67:38253
I20260810 07:15:37.272737 26812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.67:38253 every 8 connection(s)
I20260810 07:15:37.273146 26674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
I20260810 07:15:37.278352 26368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:37.273947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:51439 (local address 127.15.217.126:35287)
0810 07:15:37.274011 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:37.274013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:37.274246 (+ 233us) server_negotiation.cc:407] Connection header received
0810 07:15:37.274769 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:37.274770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:37.274780 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:37.274843 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:37.275503 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:37.275985 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:37.277045 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:37.277246 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:37.277302 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:37.278115 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:37.278116 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:37.278117 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:37.278121 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:37.278136 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:37.278140 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:37.278184 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:37.278269 (+ 85us) server_negotiation.cc:299] Negotiation successful
0810 07:15:37.278292 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 07:15:37.278601 26815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:37.273919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35287 (local address 127.15.217.67:51439)
0810 07:15:37.274112 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:37.274154 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:37.274681 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:37.274898 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:37.274901 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:37.274909 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:37.275432 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:37.275436 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:37.276076 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:37.276077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:37.276969 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:37.276972 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:37.277324 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:37.277325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:37.277441 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:37.277966 (+ 525us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:15:37.277967 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:37.277997 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:15:37.278020 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:37.278196 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:37.278201 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:37.278203 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 07:15:37.278258 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 07:15:37.278338 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
I20260810 07:15:37.279206 26813 heartbeater.cc:347] Connected to a master server at 127.15.217.126:35287
I20260810 07:15:37.279289 26813 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:37.279434 26813 heartbeater.cc:511] Master 127.15.217.126:35287 requested a full tablet report, sending...
I20260810 07:15:37.279819 26326 ts_manager.cc:194] Registered new tserver with Master: 926eda0e3a7a4f7496980f84ef3d815b (127.15.217.67:38253)
I20260810 07:15:37.279968 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 26674
I20260810 07:15:37.280048 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance
I20260810 07:15:37.280259 16229 external_mini_cluster.cc:1428] Setting key 7329a468b840cffbc4de1890c4aaac12
I20260810 07:15:37.280351 26326 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.217.67:51439
I20260810 07:15:37.281276 16229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 07:15:37.283095 26368 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:37.282672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59130 (local address 127.15.217.126:35287)
0810 07:15:37.282728 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:37.282730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:37.282835 (+ 105us) server_negotiation.cc:407] Connection header received
0810 07:15:37.282874 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:37.282874 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:37.282883 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:37.282933 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:37.283031 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59130: BlockingRecv error: recv got EOF from 127.0.0.1:59130 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 07:15:37.283167 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 26389
I20260810 07:15:37.297358 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 26533
I20260810 07:15:37.304425 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 26674
I20260810 07:15:37.311414 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 26291
2026-08-10T07:15:37Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (633 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-10T07:15:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:15:37Z Disabled control of system clock
I20260810 07:15:37.342072 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:39715
--webserver_interface=127.15.217.126
--webserver_port=0
--builtin_ntp_servers=127.15.217.84:45935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:39715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 07:15:37.427994 26825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:37.428220 26825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:37.428241 26825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:37.429435 26825 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:37.429479 26825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:37.429492 26825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:37.429504 26825 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:37.429517 26825 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:15:37.430902 26825 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:45935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:39715
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:39715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:37.431196 26825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:37.431519 26825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:37.436509 26833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:37.436573 26830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:37.436528 26831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:37.436800 26825 server_base.cc:1034] running on GCE node
I20260810 07:15:37.437076 26825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:37.437389 26825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:37.438545 26825 hybrid_clock.cc:648] HybridClock initialized: now 1786346137438522 us; error 41 us; skew 500 ppm
I20260810 07:15:37.439821 26825 webserver.cc:460] Webserver started at http://127.15.217.126:34345/ using document root <none> and password file <none>
I20260810 07:15:37.440089 26825 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:37.440173 26825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:37.440332 26825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:37.441500 26825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "9cdc25c130f44b14aa3845dfac6b4d29"
format_stamp: "Formatted at 2026-08-10 07:15:37 on dist-test-slave-10pc"
server_key: "04b2a68d2e15ce924a60464d84ed3b01"
server_key_iv: "28003713150e5c548cbec41216f235b2"
server_key_version: "encryptionkey@0"
I20260810 07:15:37.441910 26825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "9cdc25c130f44b14aa3845dfac6b4d29"
format_stamp: "Formatted at 2026-08-10 07:15:37 on dist-test-slave-10pc"
server_key: "04b2a68d2e15ce924a60464d84ed3b01"
server_key_iv: "28003713150e5c548cbec41216f235b2"
server_key_version: "encryptionkey@0"
I20260810 07:15:37.443279 26825 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:15:37.444007 26840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:37.444186 26825 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 07:15:37.444293 26825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "9cdc25c130f44b14aa3845dfac6b4d29"
format_stamp: "Formatted at 2026-08-10 07:15:37 on dist-test-slave-10pc"
server_key: "04b2a68d2e15ce924a60464d84ed3b01"
server_key_iv: "28003713150e5c548cbec41216f235b2"
server_key_version: "encryptionkey@0"
I20260810 07:15:37.444423 26825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:37.448185 26825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:37.448526 26825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:37.448693 26825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:37.453099 26825 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:39715
I20260810 07:15:37.453135 26895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:39715 every 8 connection(s)
I20260810 07:15:37.453505 26825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:15:37.454342 26896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:37.457111 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 26825
I20260810 07:15:37.457196 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:15:37.457422 16229 external_mini_cluster.cc:1428] Setting key 2e988ca7043fe4b8604a6c67aec7112b
I20260810 07:15:37.457492 26896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cdc25c130f44b14aa3845dfac6b4d29: Bootstrap starting.
I20260810 07:15:37.458324 26896 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9cdc25c130f44b14aa3845dfac6b4d29: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:37.458681 26896 log.cc:824] T 00000000000000000000000000000000 P 9cdc25c130f44b14aa3845dfac6b4d29: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:37.459501 26896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cdc25c130f44b14aa3845dfac6b4d29: No bootstrap required, opened a new log
I20260810 07:15:37.460345 26899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:37.457980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57028 (local address 127.15.217.126:39715)
0810 07:15:37.458208 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:37.458216 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:37.458263 (+ 47us) server_negotiation.cc:407] Connection header received
0810 07:15:37.458907 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:37.458917 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:37.458980 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:37.459143 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:37.459889 (+ 746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:37.459893 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:37.459894 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:37.459899 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:37.459951 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:37.459962 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:37.460058 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:37.460117 (+ 59us) server_negotiation.cc:299] Negotiation successful
0810 07:15:37.460179 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":91,"threads_started":1}
I20260810 07:15:37.461337 26896 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9cdc25c130f44b14aa3845dfac6b4d29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cdc25c130f44b14aa3845dfac6b4d29" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 39715 } }
I20260810 07:15:37.461503 26896 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9cdc25c130f44b14aa3845dfac6b4d29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:37.461558 26896 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9cdc25c130f44b14aa3845dfac6b4d29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cdc25c130f44b14aa3845dfac6b4d29, State: Initialized, Role: FOLLOWER
I20260810 07:15:37.461683 26896 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9cdc25c130f44b14aa3845dfac6b4d29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cdc25c130f44b14aa3845dfac6b4d29" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 39715 } }
I20260810 07:15:37.461781 26896 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9cdc25c130f44b14aa3845dfac6b4d29 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:15:37.461826 26896 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9cdc25c130f44b14aa3845dfac6b4d29 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:15:37.461879 26896 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9cdc25c130f44b14aa3845dfac6b4d29 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:37.462812 26896 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9cdc25c130f44b14aa3845dfac6b4d29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cdc25c130f44b14aa3845dfac6b4d29" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 39715 } }
I20260810 07:15:37.462944 26896 leader_election.cc:302] T 00000000000000000000000000000000 P 9cdc25c130f44b14aa3845dfac6b4d29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9cdc25c130f44b14aa3845dfac6b4d29; no voters:
I20260810 07:15:37.463140 26896 leader_election.cc:288] T 00000000000000000000000000000000 P 9cdc25c130f44b14aa3845dfac6b4d29 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:15:37.463220 26901 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9cdc25c130f44b14aa3845dfac6b4d29 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:37.463372 26901 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9cdc25c130f44b14aa3845dfac6b4d29 [term 1 LEADER]: Becoming Leader. State: Replica: 9cdc25c130f44b14aa3845dfac6b4d29, State: Running, Role: LEADER
I20260810 07:15:37.463462 26901 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9cdc25c130f44b14aa3845dfac6b4d29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cdc25c130f44b14aa3845dfac6b4d29" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 39715 } }
I20260810 07:15:37.463466 26896 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9cdc25c130f44b14aa3845dfac6b4d29 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:37.463922 26903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cdc25c130f44b14aa3845dfac6b4d29 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9cdc25c130f44b14aa3845dfac6b4d29. Latest consensus state: current_term: 1 leader_uuid: "9cdc25c130f44b14aa3845dfac6b4d29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cdc25c130f44b14aa3845dfac6b4d29" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 39715 } } }
I20260810 07:15:37.463927 26902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cdc25c130f44b14aa3845dfac6b4d29 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9cdc25c130f44b14aa3845dfac6b4d29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cdc25c130f44b14aa3845dfac6b4d29" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 39715 } } }
I20260810 07:15:37.464151 26903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cdc25c130f44b14aa3845dfac6b4d29 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:37.464162 26902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cdc25c130f44b14aa3845dfac6b4d29 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:37.464622 26909 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:37.465214 26909 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:15:37.466784 26909 catalog_manager.cc:1347] Generated new cluster ID: dc105c94286d438489d03d71a111ae6c
I20260810 07:15:37.466840 26909 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:15:37.499943 26909 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:15:37.500185 26909 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:37.503623 26909 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9cdc25c130f44b14aa3845dfac6b4d29: Generated new TSK 0
I20260810 07:15:37.503839 26909 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:15:37.511801 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.65:0
--local_ip_for_outbound_sockets=127.15.217.65
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:39715
--builtin_ntp_servers=127.15.217.84:45935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 07:15:37.606755 26922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:37.607018 26922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:37.607046 26922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:37.608515 26922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:37.608560 26922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:37.608623 26922 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.65
I20260810 07:15:37.610198 26922 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:45935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:39715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.65
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:37.610497 26922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:37.610849 26922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:37.616698 26933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:37.616753 26922 server_base.cc:1034] running on GCE node
W20260810 07:15:37.616716 26929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:37.616720 26931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:37.617257 26922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:37.617516 26922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:37.618685 26922 hybrid_clock.cc:648] HybridClock initialized: now 1786346137618664 us; error 63 us; skew 500 ppm
I20260810 07:15:37.620090 26922 webserver.cc:460] Webserver started at http://127.15.217.65:40999/ using document root <none> and password file <none>
I20260810 07:15:37.620383 26922 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:37.620460 26922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:37.620656 26922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:37.621796 26922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/data/instance:
uuid: "f866f57087f64c8f9c198d1b026f2620"
format_stamp: "Formatted at 2026-08-10 07:15:37 on dist-test-slave-10pc"
server_key: "33a7a07092ed9329bba02c18cfbf89cc"
server_key_iv: "d4b4bf9e907bbb9ef01dd2cf26b9d604"
server_key_version: "encryptionkey@0"
I20260810 07:15:37.622180 26922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance:
uuid: "f866f57087f64c8f9c198d1b026f2620"
format_stamp: "Formatted at 2026-08-10 07:15:37 on dist-test-slave-10pc"
server_key: "33a7a07092ed9329bba02c18cfbf89cc"
server_key_iv: "d4b4bf9e907bbb9ef01dd2cf26b9d604"
server_key_version: "encryptionkey@0"
I20260810 07:15:37.623554 26922 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:15:37.624323 26939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:37.624569 26922 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:15:37.624667 26922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/wal
uuid: "f866f57087f64c8f9c198d1b026f2620"
format_stamp: "Formatted at 2026-08-10 07:15:37 on dist-test-slave-10pc"
server_key: "33a7a07092ed9329bba02c18cfbf89cc"
server_key_iv: "d4b4bf9e907bbb9ef01dd2cf26b9d604"
server_key_version: "encryptionkey@0"
I20260810 07:15:37.624754 26922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:37.628134 26922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:37.628410 26922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:37.628579 26922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:37.628815 26922 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:37.629138 26922 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:37.629194 26922 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:37.629233 26922 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:37.629261 26922 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:37.635461 26922 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.65:42777
I20260810 07:15:37.635540 27060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.65:42777 every 8 connection(s)
I20260810 07:15:37.635880 26922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
I20260810 07:15:37.638330 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 26922
I20260810 07:15:37.638453 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance
I20260810 07:15:37.638718 16229 external_mini_cluster.cc:1428] Setting key 198d8a5ab8c7b903918a0632e595a3e6
I20260810 07:15:37.639453 26899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:37.636721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:58919 (local address 127.15.217.126:39715)
0810 07:15:37.636791 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:37.636793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:37.637101 (+ 308us) server_negotiation.cc:407] Connection header received
0810 07:15:37.637648 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:37.637650 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:37.637657 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:37.637705 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:37.639158 (+ 1453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:37.639159 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:37.639160 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:37.639164 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:37.639183 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:37.639193 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:37.639254 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:37.639367 (+ 113us) server_negotiation.cc:299] Negotiation successful
0810 07:15:37.639383 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 07:15:37.639667 27065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:37.636703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:39715 (local address 127.15.217.65:58919)
0810 07:15:37.636939 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:37.636975 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:37.637526 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:37.637787 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:37.637790 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:37.637798 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:37.638925 (+ 1127us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:15:37.638926 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:37.638957 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:15:37.638987 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:37.639267 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:37.639273 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:37.639276 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:15:37.639365 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 07:15:37.639432 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":111,"threads_started":1}
I20260810 07:15:37.640087 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.66:0
--local_ip_for_outbound_sockets=127.15.217.66
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:39715
--builtin_ntp_servers=127.15.217.84:45935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260810 07:15:37.640779 27061 heartbeater.cc:347] Connected to a master server at 127.15.217.126:39715
I20260810 07:15:37.640887 27061 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:37.641040 27061 heartbeater.cc:511] Master 127.15.217.126:39715 requested a full tablet report, sending...
I20260810 07:15:37.641650 26859 ts_manager.cc:194] Registered new tserver with Master: f866f57087f64c8f9c198d1b026f2620 (127.15.217.65:42777)
W20260810 07:15:37.727552 27066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:37.727762 27066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:37.727785 27066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:37.729162 27066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:37.729210 27066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:37.729238 27066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.66
I20260810 07:15:37.730646 27066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:45935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:39715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.66
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:37.730922 27066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:37.731288 27066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:37.736408 27075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:37.736521 27072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:37.736418 27071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:37.736686 27066 server_base.cc:1034] running on GCE node
I20260810 07:15:37.736994 27066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:37.737265 27066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:37.738435 27066 hybrid_clock.cc:648] HybridClock initialized: now 1786346137738425 us; error 35 us; skew 500 ppm
I20260810 07:15:37.739785 27066 webserver.cc:460] Webserver started at http://127.15.217.66:44671/ using document root <none> and password file <none>
I20260810 07:15:37.740080 27066 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:37.740159 27066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:37.740319 27066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:37.741528 27066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/data/instance:
uuid: "3e120632821e4deaaa572c94c76cd3b0"
format_stamp: "Formatted at 2026-08-10 07:15:37 on dist-test-slave-10pc"
server_key: "51bb2f9285ed0f85f28c3fbc9cc47994"
server_key_iv: "f76818a222902cd305b2ff03594db284"
server_key_version: "encryptionkey@0"
I20260810 07:15:37.741921 27066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance:
uuid: "3e120632821e4deaaa572c94c76cd3b0"
format_stamp: "Formatted at 2026-08-10 07:15:37 on dist-test-slave-10pc"
server_key: "51bb2f9285ed0f85f28c3fbc9cc47994"
server_key_iv: "f76818a222902cd305b2ff03594db284"
server_key_version: "encryptionkey@0"
I20260810 07:15:37.743292 27066 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:15:37.744017 27081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:37.744194 27066 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 07:15:37.744292 27066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/wal
uuid: "3e120632821e4deaaa572c94c76cd3b0"
format_stamp: "Formatted at 2026-08-10 07:15:37 on dist-test-slave-10pc"
server_key: "51bb2f9285ed0f85f28c3fbc9cc47994"
server_key_iv: "f76818a222902cd305b2ff03594db284"
server_key_version: "encryptionkey@0"
I20260810 07:15:37.744385 27066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:37.747970 27066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:37.748274 27066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:37.748427 27066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:37.748682 27066 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:37.749017 27066 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:37.749084 27066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:37.749126 27066 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:37.749153 27066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:37.755069 27066 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.66:34305
I20260810 07:15:37.755117 27203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.66:34305 every 8 connection(s)
I20260810 07:15:37.755492 27066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
I20260810 07:15:37.756265 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 27066
I20260810 07:15:37.756379 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance
I20260810 07:15:37.756649 16229 external_mini_cluster.cc:1428] Setting key 7b9105b8afc725afd8a61596b6ee53be
I20260810 07:15:37.757846 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.67:0
--local_ip_for_outbound_sockets=127.15.217.67
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:39715
--builtin_ntp_servers=127.15.217.84:45935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260810 07:15:37.759302 26899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:37.756343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:40397 (local address 127.15.217.126:39715)
0810 07:15:37.756402 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:37.756404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:37.756745 (+ 341us) server_negotiation.cc:407] Connection header received
0810 07:15:37.757412 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:37.757413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:37.757418 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:37.757488 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:37.759043 (+ 1555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:37.759044 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:37.759044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:37.759049 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:37.759066 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:37.759071 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:37.759124 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:37.759233 (+ 109us) server_negotiation.cc:299] Negotiation successful
0810 07:15:37.759246 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 07:15:37.759568 27206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:37.756335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:39715 (local address 127.15.217.66:40397)
0810 07:15:37.756582 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:37.756633 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:37.757277 (+ 644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:37.757589 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:37.757594 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:37.757605 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:37.758857 (+ 1252us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:15:37.758858 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:37.758899 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:15:37.758939 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:37.759145 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:37.759152 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:37.759155 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:15:37.759236 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 07:15:37.759306 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":125,"threads_started":1}
I20260810 07:15:37.760259 27204 heartbeater.cc:347] Connected to a master server at 127.15.217.126:39715
I20260810 07:15:37.760370 27204 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:37.760567 27204 heartbeater.cc:511] Master 127.15.217.126:39715 requested a full tablet report, sending...
I20260810 07:15:37.760965 26859 ts_manager.cc:194] Registered new tserver with Master: 3e120632821e4deaaa572c94c76cd3b0 (127.15.217.66:34305)
W20260810 07:15:37.844597 27207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:37.844832 27207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:37.844871 27207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:37.846351 27207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:37.846414 27207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:37.846459 27207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.67
I20260810 07:15:37.848037 27207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:45935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:39715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.67
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:37.848331 27207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:37.848716 27207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:37.854072 27212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:37.854148 27215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:37.854148 27213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:37.854666 27207 server_base.cc:1034] running on GCE node
I20260810 07:15:37.854920 27207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:37.855244 27207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:37.856421 27207 hybrid_clock.cc:648] HybridClock initialized: now 1786346137856413 us; error 41 us; skew 500 ppm
I20260810 07:15:37.857806 27207 webserver.cc:460] Webserver started at http://127.15.217.67:38989/ using document root <none> and password file <none>
I20260810 07:15:37.858135 27207 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:37.858227 27207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:37.858387 27207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:37.859572 27207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/data/instance:
uuid: "a629837f73dd47679836c3e296edf4ec"
format_stamp: "Formatted at 2026-08-10 07:15:37 on dist-test-slave-10pc"
server_key: "538719a6d3883daee18fb13e97b222f5"
server_key_iv: "9411895fb83d48e9e258ac4a9e46c468"
server_key_version: "encryptionkey@0"
I20260810 07:15:37.859962 27207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance:
uuid: "a629837f73dd47679836c3e296edf4ec"
format_stamp: "Formatted at 2026-08-10 07:15:37 on dist-test-slave-10pc"
server_key: "538719a6d3883daee18fb13e97b222f5"
server_key_iv: "9411895fb83d48e9e258ac4a9e46c468"
server_key_version: "encryptionkey@0"
I20260810 07:15:37.861397 27207 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:15:37.862154 27223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:37.862327 27207 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 07:15:37.862443 27207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/wal
uuid: "a629837f73dd47679836c3e296edf4ec"
format_stamp: "Formatted at 2026-08-10 07:15:37 on dist-test-slave-10pc"
server_key: "538719a6d3883daee18fb13e97b222f5"
server_key_iv: "9411895fb83d48e9e258ac4a9e46c468"
server_key_version: "encryptionkey@0"
I20260810 07:15:37.862524 27207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:37.866398 27207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:37.866688 27207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:37.866837 27207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:37.867059 27207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:37.867379 27207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:37.867440 27207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:37.867476 27207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:37.867503 27207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:37.873559 27207 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.67:41639
I20260810 07:15:37.873605 27348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.67:41639 every 8 connection(s)
I20260810 07:15:37.873953 27207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
I20260810 07:15:37.874812 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 27207
I20260810 07:15:37.874918 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance
I20260810 07:15:37.875171 16229 external_mini_cluster.cc:1428] Setting key 79ad338cf9a21784cba59b14bd9808df
I20260810 07:15:37.877439 26899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:37.874660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:45837 (local address 127.15.217.126:39715)
0810 07:15:37.874749 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:37.874752 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:37.875103 (+ 351us) server_negotiation.cc:407] Connection header received
0810 07:15:37.875851 (+ 748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:37.875853 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:37.875860 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:37.875912 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:37.877214 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:37.877215 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:37.877216 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:37.877220 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:37.877236 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:37.877240 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:37.877287 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:37.877372 (+ 85us) server_negotiation.cc:299] Negotiation successful
0810 07:15:37.877385 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 07:15:37.877677 27351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:37.874661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:39715 (local address 127.15.217.67:45837)
0810 07:15:37.874901 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:37.874954 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:37.875741 (+ 787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:37.876000 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:37.876004 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:37.876021 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:37.877067 (+ 1046us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:15:37.877068 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:37.877098 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:15:37.877127 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:37.877300 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:37.877305 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:37.877308 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:15:37.877368 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 07:15:37.877421 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":106,"threads_started":1}
I20260810 07:15:37.878175 27349 heartbeater.cc:347] Connected to a master server at 127.15.217.126:39715
I20260810 07:15:37.878293 27349 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:37.878450 27349 heartbeater.cc:511] Master 127.15.217.126:39715 requested a full tablet report, sending...
I20260810 07:15:37.878849 26859 ts_manager.cc:194] Registered new tserver with Master: a629837f73dd47679836c3e296edf4ec (127.15.217.67:41639)
I20260810 07:15:37.887364 16229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 07:15:37.889484 26899 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:37.888883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57042 (local address 127.15.217.126:39715)
0810 07:15:37.888961 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:37.888963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:37.889099 (+ 136us) server_negotiation.cc:407] Connection header received
0810 07:15:37.889194 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:37.889195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:37.889200 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:37.889247 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:37.889401 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57042: BlockingRecv error: recv got EOF from 127.0.0.1:57042 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 07:15:37.889557 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 26922
I20260810 07:15:37.897511 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 27066
I20260810 07:15:37.906379 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 27207
I20260810 07:15:37.913375 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 26825
2026-08-10T07:15:37Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (603 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:15:37 dist-test-slave-10pc krb5kdc[27359](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:15:37 dist-test-slave-10pc krb5kdc[27359](info): set up 2 sockets
Aug 10 07:15:37 dist-test-slave-10pc krb5kdc[27359](info): commencing operation
krb5kdc: starting...
W20260810 07:15:39.011929 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:15:39 dist-test-slave-10pc krb5kdc[27359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:15:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:15:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
I20260810 07:15:39.141726 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:44817
--webserver_interface=127.15.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:40901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:44817
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:39.228055 27374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:39.228349 27374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:39.228394 27374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:39.229643 27374 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:39.229722 27374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:39.229748 27374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:39.229777 27374 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:39.229807 27374 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:15:39.231361 27374 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:40901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:44817
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:44817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:39.231660 27374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:39.231913 27374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:39.237466 27382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:39.237514 27380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:39.237484 27379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:39.237766 27374 server_base.cc:1034] running on GCE node
I20260810 07:15:39.238046 27374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:39.238384 27374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:39.239574 27374 hybrid_clock.cc:648] HybridClock initialized: now 1786346139239565 us; error 53 us; skew 500 ppm
Aug 10 07:15:39 dist-test-slave-10pc krb5kdc[27359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346139, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:39.242632 27374 init.cc:377] Logged in from keytab as kudu/127.15.217.126@KRBTEST.COM (short username kudu)
I20260810 07:15:39.243161 27374 webserver.cc:460] Webserver started at http://127.15.217.126:40641/ using document root <none> and password file <none>
I20260810 07:15:39.243399 27374 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:39.243470 27374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:39.243633 27374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:39.244881 27374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "669cb575a5d64842884330f7db404f6a"
format_stamp: "Formatted at 2026-08-10 07:15:39 on dist-test-slave-10pc"
server_key: "9e14aa07149981b5bf69791c91b6e6c5"
server_key_iv: "361173d8f75ee165334403dd95a14bef"
server_key_version: "encryptionkey@0"
I20260810 07:15:39.245301 27374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "669cb575a5d64842884330f7db404f6a"
format_stamp: "Formatted at 2026-08-10 07:15:39 on dist-test-slave-10pc"
server_key: "9e14aa07149981b5bf69791c91b6e6c5"
server_key_iv: "361173d8f75ee165334403dd95a14bef"
server_key_version: "encryptionkey@0"
I20260810 07:15:39.246771 27374 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:15:39.247674 27390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:39.247908 27374 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:39.248024 27374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "669cb575a5d64842884330f7db404f6a"
format_stamp: "Formatted at 2026-08-10 07:15:39 on dist-test-slave-10pc"
server_key: "9e14aa07149981b5bf69791c91b6e6c5"
server_key_iv: "361173d8f75ee165334403dd95a14bef"
server_key_version: "encryptionkey@0"
I20260810 07:15:39.248131 27374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:39.269871 27374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:39.272255 27374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:39.272495 27374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:39.277115 27374 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:44817
I20260810 07:15:39.277182 27448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:44817 every 8 connection(s)
I20260810 07:15:39.277585 27374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:15:39.278120 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 27374
I20260810 07:15:39.278251 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:15:39.278543 16229 external_mini_cluster.cc:1428] Setting key b43e802d3eb3ab9f95435336bb9cccef
I20260810 07:15:39.279021 27449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:39.282696 27449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 669cb575a5d64842884330f7db404f6a: Bootstrap starting.
I20260810 07:15:39.283401 27449 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 669cb575a5d64842884330f7db404f6a: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:39.283831 27449 log.cc:824] T 00000000000000000000000000000000 P 669cb575a5d64842884330f7db404f6a: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:39.284780 27449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 669cb575a5d64842884330f7db404f6a: No bootstrap required, opened a new log
Aug 10 07:15:39 dist-test-slave-10pc krb5kdc[27359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:15:39.287128 27449 raft_consensus.cc:348] T 00000000000000000000000000000000 P 669cb575a5d64842884330f7db404f6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "669cb575a5d64842884330f7db404f6a" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 44817 } }
I20260810 07:15:39.287322 27449 raft_consensus.cc:374] T 00000000000000000000000000000000 P 669cb575a5d64842884330f7db404f6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:39.287369 27449 raft_consensus.cc:729] T 00000000000000000000000000000000 P 669cb575a5d64842884330f7db404f6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 669cb575a5d64842884330f7db404f6a, State: Initialized, Role: FOLLOWER
I20260810 07:15:39.287518 27449 consensus_queue.cc:260] T 00000000000000000000000000000000 P 669cb575a5d64842884330f7db404f6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "669cb575a5d64842884330f7db404f6a" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 44817 } }
I20260810 07:15:39.287595 27449 raft_consensus.cc:388] T 00000000000000000000000000000000 P 669cb575a5d64842884330f7db404f6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:15:39.287619 27449 raft_consensus.cc:482] T 00000000000000000000000000000000 P 669cb575a5d64842884330f7db404f6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:15:39.287700 27449 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 669cb575a5d64842884330f7db404f6a [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:39.288285 27449 raft_consensus.cc:504] T 00000000000000000000000000000000 P 669cb575a5d64842884330f7db404f6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "669cb575a5d64842884330f7db404f6a" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 44817 } }
I20260810 07:15:39.288388 27449 leader_election.cc:302] T 00000000000000000000000000000000 P 669cb575a5d64842884330f7db404f6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 669cb575a5d64842884330f7db404f6a; no voters:
I20260810 07:15:39.288638 27449 leader_election.cc:288] T 00000000000000000000000000000000 P 669cb575a5d64842884330f7db404f6a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:15:39.288679 27454 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 669cb575a5d64842884330f7db404f6a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:39.288818 27454 raft_consensus.cc:686] T 00000000000000000000000000000000 P 669cb575a5d64842884330f7db404f6a [term 1 LEADER]: Becoming Leader. State: Replica: 669cb575a5d64842884330f7db404f6a, State: Running, Role: LEADER
I20260810 07:15:39.288959 27449 sys_catalog.cc:564] T 00000000000000000000000000000000 P 669cb575a5d64842884330f7db404f6a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:39.288980 27454 consensus_queue.cc:237] T 00000000000000000000000000000000 P 669cb575a5d64842884330f7db404f6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "669cb575a5d64842884330f7db404f6a" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 44817 } }
I20260810 07:15:39.289359 27454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 669cb575a5d64842884330f7db404f6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 669cb575a5d64842884330f7db404f6a. Latest consensus state: current_term: 1 leader_uuid: "669cb575a5d64842884330f7db404f6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "669cb575a5d64842884330f7db404f6a" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 44817 } } }
I20260810 07:15:39.289379 27455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 669cb575a5d64842884330f7db404f6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "669cb575a5d64842884330f7db404f6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "669cb575a5d64842884330f7db404f6a" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 44817 } } }
I20260810 07:15:39.289461 27455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 669cb575a5d64842884330f7db404f6a [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:39.289446 27454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 669cb575a5d64842884330f7db404f6a [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:39.290061 27461 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:39.290750 27461 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:15:39.290887 27452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.279183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58278 (local address 127.15.217.126:44817)
0810 07:15:39.279369 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:39.279383 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:39.279410 (+ 27us) server_negotiation.cc:407] Connection header received
0810 07:15:39.279882 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:39.279889 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:39.279937 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:39.280058 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:39.280947 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.281857 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.282953 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.283386 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.283505 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.286355 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:39.286360 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:39.286360 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:39.286372 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:39.288366 (+ 1994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:39.288846 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:39.288849 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:39.288850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:39.288963 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:39.289310 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:39.289311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:39.289312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:39.289782 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:39.289927 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:39.290476 (+ 549us) server_negotiation.cc:299] Negotiation successful
0810 07:15:39.290634 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":97,"threads_started":1}
I20260810 07:15:39.292362 27461 catalog_manager.cc:1347] Generated new cluster ID: 638306e2f55b4399a2ebeecacdc49ff5
I20260810 07:15:39.292416 27461 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:15:39.304615 27461 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:15:39.305233 27461 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:39.310560 27461 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 669cb575a5d64842884330f7db404f6a: Generated new TSK 0
I20260810 07:15:39.310804 27461 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab.
Entry for principal kudu/127.15.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab.
I20260810 07:15:39.341224 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.65:0
--local_ip_for_outbound_sockets=127.15.217.65
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:44817
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:40901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:39.430966 27476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:39.431198 27476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:39.431226 27476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:39.432524 27476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:39.432572 27476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:39.432600 27476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.65
I20260810 07:15:39.434134 27476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:40901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:44817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.65
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:39.434419 27476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:39.434684 27476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:39.440174 27484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:39.440191 27481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:39.440188 27482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:39.440706 27476 server_base.cc:1034] running on GCE node
I20260810 07:15:39.440969 27476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:39.441241 27476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:39.442484 27476 hybrid_clock.cc:648] HybridClock initialized: now 1786346139442470 us; error 48 us; skew 500 ppm
Aug 10 07:15:39 dist-test-slave-10pc krb5kdc[27359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346139, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:39.445413 27476 init.cc:377] Logged in from keytab as kudu/127.15.217.65@KRBTEST.COM (short username kudu)
I20260810 07:15:39.445958 27476 webserver.cc:460] Webserver started at http://127.15.217.65:43357/ using document root <none> and password file <none>
I20260810 07:15:39.446244 27476 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:39.446316 27476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:39.446477 27476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:39.447649 27476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/data/instance:
uuid: "6f135bd173b6425e81ea799289689be7"
format_stamp: "Formatted at 2026-08-10 07:15:39 on dist-test-slave-10pc"
server_key: "43e3781a7cb59e6e5eee2e90d0ad7a0b"
server_key_iv: "4b9d98f31424a99b9d9bcda309a1ae07"
server_key_version: "encryptionkey@0"
I20260810 07:15:39.448040 27476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance:
uuid: "6f135bd173b6425e81ea799289689be7"
format_stamp: "Formatted at 2026-08-10 07:15:39 on dist-test-slave-10pc"
server_key: "43e3781a7cb59e6e5eee2e90d0ad7a0b"
server_key_iv: "4b9d98f31424a99b9d9bcda309a1ae07"
server_key_version: "encryptionkey@0"
I20260810 07:15:39.449745 27476 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:15:39.450670 27491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:39.450850 27476 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:15:39.451068 27476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/wal
uuid: "6f135bd173b6425e81ea799289689be7"
format_stamp: "Formatted at 2026-08-10 07:15:39 on dist-test-slave-10pc"
server_key: "43e3781a7cb59e6e5eee2e90d0ad7a0b"
server_key_iv: "4b9d98f31424a99b9d9bcda309a1ae07"
server_key_version: "encryptionkey@0"
I20260810 07:15:39.451157 27476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:39.462172 27476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:39.469271 27476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:39.469411 27476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:39.469630 27476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:39.470000 27476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:39.470043 27476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:39.470067 27476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:39.470117 27476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:39.476428 27476 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.65:35925
I20260810 07:15:39.476513 27614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.65:35925 every 8 connection(s)
I20260810 07:15:39.476859 27476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
I20260810 07:15:39.478283 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 27476
I20260810 07:15:39.478369 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance
I20260810 07:15:39.478587 16229 external_mini_cluster.cc:1428] Setting key 69c95230569fb44474c404bafa875021
Aug 10 07:15:39 dist-test-slave-10pc krb5kdc[27359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346139, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:15:39.488883 27452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.477645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:56511 (local address 127.15.217.126:44817)
0810 07:15:39.477698 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:39.477700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:39.477979 (+ 279us) server_negotiation.cc:407] Connection header received
0810 07:15:39.478765 (+ 786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:39.478767 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:39.478781 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:39.478837 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:39.479978 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.480784 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.482288 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.482520 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.482583 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.485470 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:39.485471 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:39.485471 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:39.485478 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:39.487220 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:39.487815 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:39.487816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:39.487816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:39.487875 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:39.488178 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:39.488178 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:39.488179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:39.488289 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:39.488495 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:39.488734 (+ 239us) server_negotiation.cc:299] Negotiation successful
0810 07:15:39.488814 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 07:15:39.489192 27620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.477597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:44817 (local address 127.15.217.65:56511)
0810 07:15:39.477779 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:39.477825 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:39.478641 (+ 816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:39.478919 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:39.478924 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:39.479654 (+ 730us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:39.479867 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:39.479872 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.480895 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.480896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:39.482182 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:39.482185 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.482617 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.482618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:39.482778 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.483392 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:39.483401 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:39.485344 (+ 1943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:39.487313 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:39.487322 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:39.487325 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:39.487707 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:39.487947 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:39.487948 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:39.487948 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:39.488103 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:39.488358 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:39.488365 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:39.488574 (+ 209us) client_negotiation.cc:769] Sending connection context
0810 07:15:39.488712 (+ 138us) client_negotiation.cc:240] Negotiation successful
0810 07:15:39.488907 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":105,"threads_started":1}
WARNING: no policy specified for kudu/127.15.217.66@KRBTEST.COM; defaulting to no policy
I20260810 07:15:39.489769 27615 heartbeater.cc:347] Connected to a master server at 127.15.217.126:44817
I20260810 07:15:39.489897 27615 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:39.490090 27615 heartbeater.cc:511] Master 127.15.217.126:44817 requested a full tablet report, sending...
I20260810 07:15:39.490774 27411 ts_manager.cc:194] Registered new tserver with Master: 6f135bd173b6425e81ea799289689be7 (127.15.217.65:35925)
I20260810 07:15:39.491397 27411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.65@KRBTEST.COM'} at 127.15.217.65:56511
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab.
Entry for principal kudu/127.15.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab.
I20260810 07:15:39.507344 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.66:0
--local_ip_for_outbound_sockets=127.15.217.66
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:44817
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:40901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:39.594621 27623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:39.594832 27623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:39.594856 27623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:39.596016 27623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:39.596063 27623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:39.596091 27623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.66
I20260810 07:15:39.597599 27623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:40901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:44817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.66
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:39.597853 27623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:39.598110 27623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:39.603253 27629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:39.603263 27632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:39.603267 27630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:39.603570 27623 server_base.cc:1034] running on GCE node
I20260810 07:15:39.603777 27623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:39.604079 27623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:39.605259 27623 hybrid_clock.cc:648] HybridClock initialized: now 1786346139605257 us; error 45 us; skew 500 ppm
Aug 10 07:15:39 dist-test-slave-10pc krb5kdc[27359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346139, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:39.607882 27623 init.cc:377] Logged in from keytab as kudu/127.15.217.66@KRBTEST.COM (short username kudu)
I20260810 07:15:39.608345 27623 webserver.cc:460] Webserver started at http://127.15.217.66:33889/ using document root <none> and password file <none>
I20260810 07:15:39.608636 27623 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:39.608709 27623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:39.608872 27623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:39.610046 27623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/data/instance:
uuid: "16935ff27a0e4a40b3f59ac3d0fbe95f"
format_stamp: "Formatted at 2026-08-10 07:15:39 on dist-test-slave-10pc"
server_key: "16d75fda763b2ff9c344725c3b95acbd"
server_key_iv: "ed5528b49f2a7609fe414434bb90fa4b"
server_key_version: "encryptionkey@0"
I20260810 07:15:39.610448 27623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance:
uuid: "16935ff27a0e4a40b3f59ac3d0fbe95f"
format_stamp: "Formatted at 2026-08-10 07:15:39 on dist-test-slave-10pc"
server_key: "16d75fda763b2ff9c344725c3b95acbd"
server_key_iv: "ed5528b49f2a7609fe414434bb90fa4b"
server_key_version: "encryptionkey@0"
I20260810 07:15:39.612002 27623 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:15:39.613080 27639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:39.613255 27623 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 07:15:39.613360 27623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/wal
uuid: "16935ff27a0e4a40b3f59ac3d0fbe95f"
format_stamp: "Formatted at 2026-08-10 07:15:39 on dist-test-slave-10pc"
server_key: "16d75fda763b2ff9c344725c3b95acbd"
server_key_iv: "ed5528b49f2a7609fe414434bb90fa4b"
server_key_version: "encryptionkey@0"
I20260810 07:15:39.613452 27623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:39.627043 27623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:39.629418 27623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:39.629603 27623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:39.629873 27623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:39.630224 27623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:39.630280 27623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:39.630314 27623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:39.630358 27623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:39.636907 27623 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.66:38939
I20260810 07:15:39.637054 27762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.66:38939 every 8 connection(s)
I20260810 07:15:39.637355 27623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
I20260810 07:15:39.643690 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 27623
I20260810 07:15:39.643821 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance
Aug 10 07:15:39 dist-test-slave-10pc krb5kdc[27359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346139, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:15:39.644081 16229 external_mini_cluster.cc:1428] Setting key 3cfd75f05c1105d3e96e587611bf8697
I20260810 07:15:39.647950 27452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.638175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:47955 (local address 127.15.217.126:44817)
0810 07:15:39.638252 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:39.638254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:39.638503 (+ 249us) server_negotiation.cc:407] Connection header received
0810 07:15:39.639002 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:39.639003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:39.639014 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:39.639070 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:39.640092 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.640700 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.641757 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.641965 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.642030 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.644528 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:39.644529 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:39.644529 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:39.644537 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:39.646373 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:39.646807 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:39.646808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:39.646808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:39.646882 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:39.647252 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:39.647253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:39.647253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:39.647372 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:39.647430 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:39.647809 (+ 379us) server_negotiation.cc:299] Negotiation successful
0810 07:15:39.647883 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 07:15:39.648262 27765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.638157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:44817 (local address 127.15.217.66:47955)
0810 07:15:39.638339 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:39.638379 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:39.638907 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:39.639133 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:39.639136 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:39.639752 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:39.639973 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:39.639978 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.640810 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.640811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:39.641661 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:39.641664 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.642050 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.642051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:39.642164 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.642744 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:39.642753 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:39.644359 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:39.646463 (+ 2104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:39.646471 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:39.646473 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:39.646723 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:39.646981 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:39.646982 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:39.646983 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:39.647170 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:39.647462 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:39.647471 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:39.647658 (+ 187us) client_negotiation.cc:769] Sending connection context
0810 07:15:39.647786 (+ 128us) client_negotiation.cc:240] Negotiation successful
0810 07:15:39.647981 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":98,"threads_started":1}
I20260810 07:15:39.648918 27763 heartbeater.cc:347] Connected to a master server at 127.15.217.126:44817
I20260810 07:15:39.649052 27763 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:39.649252 27763 heartbeater.cc:511] Master 127.15.217.126:44817 requested a full tablet report, sending...
I20260810 07:15:39.649679 27411 ts_manager.cc:194] Registered new tserver with Master: 16935ff27a0e4a40b3f59ac3d0fbe95f (127.15.217.66:38939)
I20260810 07:15:39.650141 27411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.66@KRBTEST.COM'} at 127.15.217.66:47955
WARNING: no policy specified for kudu/127.15.217.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab.
Entry for principal kudu/127.15.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab.
I20260810 07:15:39.673787 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.67:0
--local_ip_for_outbound_sockets=127.15.217.67
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:44817
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:40901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:39.766714 27769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:39.767006 27769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:39.767037 27769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:39.768181 27769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:39.768217 27769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:39.768244 27769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.67
I20260810 07:15:39.769840 27769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:40901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:44817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.67
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:15:39.770069 27769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:39.770267 27769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:39.775525 27777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:39.775512 27776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:39.775512 27779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:39.775825 27769 server_base.cc:1034] running on GCE node
I20260810 07:15:39.776109 27769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:39.776361 27769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:39.777521 27769 hybrid_clock.cc:648] HybridClock initialized: now 1786346139777510 us; error 33 us; skew 500 ppm
Aug 10 07:15:39 dist-test-slave-10pc krb5kdc[27359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346139, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:39.780177 27769 init.cc:377] Logged in from keytab as kudu/127.15.217.67@KRBTEST.COM (short username kudu)
I20260810 07:15:39.780685 27769 webserver.cc:460] Webserver started at http://127.15.217.67:38785/ using document root <none> and password file <none>
I20260810 07:15:39.780902 27769 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:39.780944 27769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:39.781047 27769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:39.782163 27769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/data/instance:
uuid: "a4d0caa941be46e6afef35b541246363"
format_stamp: "Formatted at 2026-08-10 07:15:39 on dist-test-slave-10pc"
server_key: "88665c6bc67e916b14ea85b702019827"
server_key_iv: "e9a8b7d14f15cf7f631ab6170c020887"
server_key_version: "encryptionkey@0"
I20260810 07:15:39.782495 27769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance:
uuid: "a4d0caa941be46e6afef35b541246363"
format_stamp: "Formatted at 2026-08-10 07:15:39 on dist-test-slave-10pc"
server_key: "88665c6bc67e916b14ea85b702019827"
server_key_iv: "e9a8b7d14f15cf7f631ab6170c020887"
server_key_version: "encryptionkey@0"
I20260810 07:15:39.783859 27769 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:15:39.784731 27786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:39.785017 27769 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:39.785149 27769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/wal
uuid: "a4d0caa941be46e6afef35b541246363"
format_stamp: "Formatted at 2026-08-10 07:15:39 on dist-test-slave-10pc"
server_key: "88665c6bc67e916b14ea85b702019827"
server_key_iv: "e9a8b7d14f15cf7f631ab6170c020887"
server_key_version: "encryptionkey@0"
I20260810 07:15:39.785239 27769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:39.799561 27769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:39.801900 27769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:39.802079 27769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:39.802333 27769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:39.802692 27769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:39.802748 27769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:39.802790 27769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:39.802858 27769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:39.808939 27769 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.67:38919
I20260810 07:15:39.809057 27909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.67:38919 every 8 connection(s)
I20260810 07:15:39.809367 27769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
I20260810 07:15:39.810423 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 27769
I20260810 07:15:39.810542 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance
I20260810 07:15:39.810770 16229 external_mini_cluster.cc:1428] Setting key a24c7641ec54bb413ec0af9d282bb20d
Aug 10 07:15:39 dist-test-slave-10pc krb5kdc[27359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346139, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:15:39.820010 27452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.810177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:50567 (local address 127.15.217.126:44817)
0810 07:15:39.810257 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:39.810259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:39.811106 (+ 847us) server_negotiation.cc:407] Connection header received
0810 07:15:39.811743 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:39.811744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:39.811756 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:39.811808 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:39.812858 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.813399 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.814433 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.814629 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.814682 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.816956 (+ 2274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:39.816957 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:39.816957 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:39.816964 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:39.818589 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:39.818992 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:39.818993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:39.818994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:39.819064 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:39.819323 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:39.819324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:39.819324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:39.819455 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:39.819516 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:39.819852 (+ 336us) server_negotiation.cc:299] Negotiation successful
0810 07:15:39.819931 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 07:15:39.820158 27913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.810147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:44817 (local address 127.15.217.67:50567)
0810 07:15:39.810939 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:39.811004 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:39.811643 (+ 639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:39.811869 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:39.811873 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:39.812546 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:39.812772 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:39.812778 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.813514 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.813515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:39.814352 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:39.814355 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.814716 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.814717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:39.814823 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.815307 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:39.815316 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:39.816844 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:39.818670 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:39.818678 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:39.818680 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:39.818900 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:39.819149 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:39.819150 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:39.819150 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:39.819258 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:39.819549 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:39.819556 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:39.819686 (+ 130us) client_negotiation.cc:769] Sending connection context
0810 07:15:39.819790 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 07:15:39.819923 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"thread_start_us":87,"threads_started":1}
I20260810 07:15:39.820726 27910 heartbeater.cc:347] Connected to a master server at 127.15.217.126:44817
I20260810 07:15:39.820863 27910 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:39.821072 27910 heartbeater.cc:511] Master 127.15.217.126:44817 requested a full tablet report, sending...
I20260810 07:15:39.821491 27411 ts_manager.cc:194] Registered new tserver with Master: a4d0caa941be46e6afef35b541246363 (127.15.217.67:38919)
I20260810 07:15:39.821899 27411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.67@KRBTEST.COM'} at 127.15.217.67:50567
I20260810 07:15:39.822582 16229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:15:39.829816 27452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.823951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58294 (local address 127.15.217.126:44817)
0810 07:15:39.824036 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:39.824038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:39.824106 (+ 68us) server_negotiation.cc:407] Connection header received
0810 07:15:39.824157 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:39.824158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:39.824166 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:39.824212 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:39.824826 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.825322 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.826030 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.826207 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.826257 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.827378 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:39.827379 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:39.827380 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:39.827386 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:39.828792 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:39.829120 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:39.829121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:39.829121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:39.829172 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:39.829339 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:39.829339 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:39.829340 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:39.829497 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:39.829573 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:39.829663 (+ 90us) server_negotiation.cc:299] Negotiation successful
0810 07:15:39.829723 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 07:15:39.831614 27411 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58294:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:15:39.833422 27411 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:15:39.841070 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.836401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59546 (local address 127.15.217.65:35925)
0810 07:15:39.836617 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:39.836624 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:39.837060 (+ 436us) server_negotiation.cc:407] Connection header received
0810 07:15:39.837191 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:39.837194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:39.837210 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:39.837259 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:39.837515 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.838662 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.839998 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.840611 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.840708 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.840772 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:39.840921 (+ 149us) server_negotiation.cc:299] Negotiation successful
0810 07:15:39.840975 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":144,"threads_started":1}
I20260810 07:15:39.841053 27928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.836254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:35925 (local address 127.0.0.1:59546)
0810 07:15:39.836980 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:39.836992 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:39.837093 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:39.837370 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:39.837370 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:39.837372 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:39.837408 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:39.837410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.838796 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.838797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:39.839897 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:39.839899 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.840711 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.840712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:39.840793 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.840807 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 07:15:39.840881 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 07:15:39.840939 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"mutex_wait_us":2,"thread_start_us":74,"threads_started":1}
I20260810 07:15:39.841107 27926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.836501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44828 (local address 127.15.217.66:38939)
0810 07:15:39.836692 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:39.836697 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:39.836848 (+ 151us) server_negotiation.cc:407] Connection header received
0810 07:15:39.837150 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:39.837153 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:39.837171 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:39.837252 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:39.837512 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.838210 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.839870 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.840614 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.840689 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.840752 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:39.840966 (+ 214us) server_negotiation.cc:299] Negotiation successful
0810 07:15:39.841014 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":101,"threads_started":1}
I20260810 07:15:39.841364 27922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.836254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:38939 (local address 127.0.0.1:44828)
0810 07:15:39.836723 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:39.836770 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:39.836871 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:39.837344 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:39.837346 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:39.837348 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:39.837408 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:39.837411 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.838311 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.838312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:39.839748 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:39.839751 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.840791 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.840793 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:39.840861 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.840870 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 07:15:39.841238 (+ 368us) client_negotiation.cc:240] Negotiation successful
0810 07:15:39.841288 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"mutex_wait_us":3,"thread_start_us":67,"threads_started":1}
I20260810 07:15:39.841552 27923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.836782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:38919 (local address 127.0.0.1:56914)
0810 07:15:39.836941 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:39.836951 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:39.837005 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:39.837302 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:39.837306 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:39.837308 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:39.837373 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:39.837374 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.838727 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.838729 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:39.839730 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:39.839733 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.840793 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.840794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:39.840861 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.840871 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 07:15:39.840922 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 07:15:39.840983 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":113,"thread_start_us":76,"threads_started":1}
I20260810 07:15:39.841868 27927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.836621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56914 (local address 127.15.217.67:38919)
0810 07:15:39.836790 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:39.836793 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:39.837006 (+ 213us) server_negotiation.cc:407] Connection header received
0810 07:15:39.837080 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:39.837082 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:39.837121 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:39.837196 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:39.837464 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.838601 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.839832 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.840614 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.840689 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.840751 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:39.841099 (+ 348us) server_negotiation.cc:299] Negotiation successful
0810 07:15:39.841804 (+ 705us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":86,"threads_started":1}
I20260810 07:15:39.841857 27542 tablet_service.cc:1467] Processing CreateTablet for tablet 927a4a59afe843cbbee59c700a0e3083 (DEFAULT_TABLE table=test-table [id=818920f585fb4dffa696b469a73233c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:39.842291 27542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 927a4a59afe843cbbee59c700a0e3083. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:39.842585 27835 tablet_service.cc:1467] Processing CreateTablet for tablet 927a4a59afe843cbbee59c700a0e3083 (DEFAULT_TABLE table=test-table [id=818920f585fb4dffa696b469a73233c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:39.842945 27835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 927a4a59afe843cbbee59c700a0e3083. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:39.844951 27688 tablet_service.cc:1467] Processing CreateTablet for tablet 927a4a59afe843cbbee59c700a0e3083 (DEFAULT_TABLE table=test-table [id=818920f585fb4dffa696b469a73233c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:39.845335 27688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 927a4a59afe843cbbee59c700a0e3083. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:39.846181 27930 tablet_bootstrap.cc:492] T 927a4a59afe843cbbee59c700a0e3083 P 6f135bd173b6425e81ea799289689be7: Bootstrap starting.
I20260810 07:15:39.846501 27931 tablet_bootstrap.cc:492] T 927a4a59afe843cbbee59c700a0e3083 P a4d0caa941be46e6afef35b541246363: Bootstrap starting.
I20260810 07:15:39.847096 27930 tablet_bootstrap.cc:654] T 927a4a59afe843cbbee59c700a0e3083 P 6f135bd173b6425e81ea799289689be7: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:39.847132 27931 tablet_bootstrap.cc:654] T 927a4a59afe843cbbee59c700a0e3083 P a4d0caa941be46e6afef35b541246363: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:39.847654 27931 log.cc:824] T 927a4a59afe843cbbee59c700a0e3083 P a4d0caa941be46e6afef35b541246363: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:39.847901 27930 log.cc:824] T 927a4a59afe843cbbee59c700a0e3083 P 6f135bd173b6425e81ea799289689be7: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:39.848519 27931 tablet_bootstrap.cc:492] T 927a4a59afe843cbbee59c700a0e3083 P a4d0caa941be46e6afef35b541246363: No bootstrap required, opened a new log
I20260810 07:15:39.848593 27931 ts_tablet_manager.cc:1397] T 927a4a59afe843cbbee59c700a0e3083 P a4d0caa941be46e6afef35b541246363: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:39.848879 27930 tablet_bootstrap.cc:492] T 927a4a59afe843cbbee59c700a0e3083 P 6f135bd173b6425e81ea799289689be7: No bootstrap required, opened a new log
I20260810 07:15:39.848989 27930 ts_tablet_manager.cc:1397] T 927a4a59afe843cbbee59c700a0e3083 P 6f135bd173b6425e81ea799289689be7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:15:39.849380 27935 tablet_bootstrap.cc:492] T 927a4a59afe843cbbee59c700a0e3083 P 16935ff27a0e4a40b3f59ac3d0fbe95f: Bootstrap starting.
I20260810 07:15:39.850133 27935 tablet_bootstrap.cc:654] T 927a4a59afe843cbbee59c700a0e3083 P 16935ff27a0e4a40b3f59ac3d0fbe95f: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:39.850739 27935 log.cc:824] T 927a4a59afe843cbbee59c700a0e3083 P 16935ff27a0e4a40b3f59ac3d0fbe95f: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:39.850992 27931 raft_consensus.cc:348] T 927a4a59afe843cbbee59c700a0e3083 P a4d0caa941be46e6afef35b541246363 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16935ff27a0e4a40b3f59ac3d0fbe95f" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 38939 } } peers { permanent_uuid: "6f135bd173b6425e81ea799289689be7" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 35925 } } peers { permanent_uuid: "a4d0caa941be46e6afef35b541246363" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 38919 } }
I20260810 07:15:39.851153 27931 raft_consensus.cc:374] T 927a4a59afe843cbbee59c700a0e3083 P a4d0caa941be46e6afef35b541246363 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:39.851178 27931 raft_consensus.cc:729] T 927a4a59afe843cbbee59c700a0e3083 P a4d0caa941be46e6afef35b541246363 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4d0caa941be46e6afef35b541246363, State: Initialized, Role: FOLLOWER
I20260810 07:15:39.851020 27930 raft_consensus.cc:348] T 927a4a59afe843cbbee59c700a0e3083 P 6f135bd173b6425e81ea799289689be7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16935ff27a0e4a40b3f59ac3d0fbe95f" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 38939 } } peers { permanent_uuid: "6f135bd173b6425e81ea799289689be7" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 35925 } } peers { permanent_uuid: "a4d0caa941be46e6afef35b541246363" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 38919 } }
I20260810 07:15:39.851213 27930 raft_consensus.cc:374] T 927a4a59afe843cbbee59c700a0e3083 P 6f135bd173b6425e81ea799289689be7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:39.851261 27930 raft_consensus.cc:729] T 927a4a59afe843cbbee59c700a0e3083 P 6f135bd173b6425e81ea799289689be7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f135bd173b6425e81ea799289689be7, State: Initialized, Role: FOLLOWER
I20260810 07:15:39.851290 27931 consensus_queue.cc:260] T 927a4a59afe843cbbee59c700a0e3083 P a4d0caa941be46e6afef35b541246363 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16935ff27a0e4a40b3f59ac3d0fbe95f" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 38939 } } peers { permanent_uuid: "6f135bd173b6425e81ea799289689be7" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 35925 } } peers { permanent_uuid: "a4d0caa941be46e6afef35b541246363" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 38919 } }
I20260810 07:15:39.851384 27930 consensus_queue.cc:260] T 927a4a59afe843cbbee59c700a0e3083 P 6f135bd173b6425e81ea799289689be7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16935ff27a0e4a40b3f59ac3d0fbe95f" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 38939 } } peers { permanent_uuid: "6f135bd173b6425e81ea799289689be7" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 35925 } } peers { permanent_uuid: "a4d0caa941be46e6afef35b541246363" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 38919 } }
I20260810 07:15:39.851609 27931 ts_tablet_manager.cc:1428] T 927a4a59afe843cbbee59c700a0e3083 P a4d0caa941be46e6afef35b541246363: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:15:39.851735 27930 ts_tablet_manager.cc:1428] T 927a4a59afe843cbbee59c700a0e3083 P 6f135bd173b6425e81ea799289689be7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:15:39.851780 27910 heartbeater.cc:503] Master 127.15.217.126:44817 was elected leader, sending a full tablet report...
I20260810 07:15:39.851874 27615 heartbeater.cc:503] Master 127.15.217.126:44817 was elected leader, sending a full tablet report...
I20260810 07:15:39.851917 27935 tablet_bootstrap.cc:492] T 927a4a59afe843cbbee59c700a0e3083 P 16935ff27a0e4a40b3f59ac3d0fbe95f: No bootstrap required, opened a new log
I20260810 07:15:39.851974 27935 ts_tablet_manager.cc:1397] T 927a4a59afe843cbbee59c700a0e3083 P 16935ff27a0e4a40b3f59ac3d0fbe95f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:15:39.854055 27935 raft_consensus.cc:348] T 927a4a59afe843cbbee59c700a0e3083 P 16935ff27a0e4a40b3f59ac3d0fbe95f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16935ff27a0e4a40b3f59ac3d0fbe95f" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 38939 } } peers { permanent_uuid: "6f135bd173b6425e81ea799289689be7" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 35925 } } peers { permanent_uuid: "a4d0caa941be46e6afef35b541246363" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 38919 } }
I20260810 07:15:39.854185 27935 raft_consensus.cc:374] T 927a4a59afe843cbbee59c700a0e3083 P 16935ff27a0e4a40b3f59ac3d0fbe95f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:39.854206 27935 raft_consensus.cc:729] T 927a4a59afe843cbbee59c700a0e3083 P 16935ff27a0e4a40b3f59ac3d0fbe95f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16935ff27a0e4a40b3f59ac3d0fbe95f, State: Initialized, Role: FOLLOWER
I20260810 07:15:39.854338 27935 consensus_queue.cc:260] T 927a4a59afe843cbbee59c700a0e3083 P 16935ff27a0e4a40b3f59ac3d0fbe95f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16935ff27a0e4a40b3f59ac3d0fbe95f" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 38939 } } peers { permanent_uuid: "6f135bd173b6425e81ea799289689be7" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 35925 } } peers { permanent_uuid: "a4d0caa941be46e6afef35b541246363" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 38919 } }
I20260810 07:15:39.854635 27935 ts_tablet_manager.cc:1428] T 927a4a59afe843cbbee59c700a0e3083 P 16935ff27a0e4a40b3f59ac3d0fbe95f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:15:39.854732 27763 heartbeater.cc:503] Master 127.15.217.126:44817 was elected leader, sending a full tablet report...
I20260810 07:15:39.860652 27939 raft_consensus.cc:482] T 927a4a59afe843cbbee59c700a0e3083 P 16935ff27a0e4a40b3f59ac3d0fbe95f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:15:39.860771 27939 raft_consensus.cc:504] T 927a4a59afe843cbbee59c700a0e3083 P 16935ff27a0e4a40b3f59ac3d0fbe95f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16935ff27a0e4a40b3f59ac3d0fbe95f" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 38939 } } peers { permanent_uuid: "6f135bd173b6425e81ea799289689be7" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 35925 } } peers { permanent_uuid: "a4d0caa941be46e6afef35b541246363" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 38919 } }
I20260810 07:15:39.861145 27939 leader_election.cc:288] T 927a4a59afe843cbbee59c700a0e3083 P 16935ff27a0e4a40b3f59ac3d0fbe95f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f135bd173b6425e81ea799289689be7 (127.15.217.65:35925), a4d0caa941be46e6afef35b541246363 (127.15.217.67:38919)
I20260810 07:15:39.865463 27927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.861566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:51967 (local address 127.15.217.67:38919)
0810 07:15:39.861624 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:39.861626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:39.861760 (+ 134us) server_negotiation.cc:407] Connection header received
0810 07:15:39.861834 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:39.861835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:39.861846 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:39.861903 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:39.862055 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.862749 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.864387 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.865051 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.865121 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.865131 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:39.865352 (+ 221us) server_negotiation.cc:299] Negotiation successful
0810 07:15:39.865394 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 07:15:39.865517 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.861386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:46705 (local address 127.15.217.65:35925)
0810 07:15:39.861434 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:39.861437 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:39.861541 (+ 104us) server_negotiation.cc:407] Connection header received
0810 07:15:39.861576 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:39.861577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:39.861586 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:39.861653 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:39.861859 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.862644 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.864387 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.865032 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.865121 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.865131 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:39.865354 (+ 223us) server_negotiation.cc:299] Negotiation successful
0810 07:15:39.865394 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 07:15:39.865499 27765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.861330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:35925 (local address 127.15.217.66:46705)
0810 07:15:39.861410 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:39.861426 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:39.861507 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:39.861675 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:39.861676 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:39.861678 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:39.861783 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:39.861785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.862749 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.862750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:39.864284 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:39.864286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.865154 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.865155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:39.865265 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.865272 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 07:15:39.865341 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 07:15:39.865396 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":104}
I20260810 07:15:39.865621 27941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.861430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:38919 (local address 127.15.217.66:51967)
0810 07:15:39.861676 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:39.861687 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:39.861754 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:39.861921 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:39.861922 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:39.861923 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:39.861985 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:39.861986 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.862830 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.862831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:39.864234 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:39.864238 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.865154 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.865155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:39.865265 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.865272 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 07:15:39.865341 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 07:15:39.865396 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":44,"thread_start_us":103,"threads_started":1}
I20260810 07:15:39.865989 27856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "927a4a59afe843cbbee59c700a0e3083" candidate_uuid: "16935ff27a0e4a40b3f59ac3d0fbe95f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4d0caa941be46e6afef35b541246363" is_pre_election: true
I20260810 07:15:39.865967 27566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "927a4a59afe843cbbee59c700a0e3083" candidate_uuid: "16935ff27a0e4a40b3f59ac3d0fbe95f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f135bd173b6425e81ea799289689be7" is_pre_election: true
I20260810 07:15:39.866163 27856 raft_consensus.cc:2454] T 927a4a59afe843cbbee59c700a0e3083 P a4d0caa941be46e6afef35b541246363 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16935ff27a0e4a40b3f59ac3d0fbe95f in term 0.
I20260810 07:15:39.866164 27566 raft_consensus.cc:2454] T 927a4a59afe843cbbee59c700a0e3083 P 6f135bd173b6425e81ea799289689be7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16935ff27a0e4a40b3f59ac3d0fbe95f in term 0.
I20260810 07:15:39.866412 27643 leader_election.cc:302] T 927a4a59afe843cbbee59c700a0e3083 P 16935ff27a0e4a40b3f59ac3d0fbe95f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16935ff27a0e4a40b3f59ac3d0fbe95f, 6f135bd173b6425e81ea799289689be7; no voters:
I20260810 07:15:39.866568 27939 raft_consensus.cc:2781] T 927a4a59afe843cbbee59c700a0e3083 P 16935ff27a0e4a40b3f59ac3d0fbe95f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:15:39.866648 27939 raft_consensus.cc:482] T 927a4a59afe843cbbee59c700a0e3083 P 16935ff27a0e4a40b3f59ac3d0fbe95f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:15:39.866683 27939 raft_consensus.cc:3037] T 927a4a59afe843cbbee59c700a0e3083 P 16935ff27a0e4a40b3f59ac3d0fbe95f [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:39.867473 27939 raft_consensus.cc:504] T 927a4a59afe843cbbee59c700a0e3083 P 16935ff27a0e4a40b3f59ac3d0fbe95f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16935ff27a0e4a40b3f59ac3d0fbe95f" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 38939 } } peers { permanent_uuid: "6f135bd173b6425e81ea799289689be7" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 35925 } } peers { permanent_uuid: "a4d0caa941be46e6afef35b541246363" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 38919 } }
I20260810 07:15:39.867619 27939 leader_election.cc:288] T 927a4a59afe843cbbee59c700a0e3083 P 16935ff27a0e4a40b3f59ac3d0fbe95f [CANDIDATE]: Term 1 election: Requested vote from peers 6f135bd173b6425e81ea799289689be7 (127.15.217.65:35925), a4d0caa941be46e6afef35b541246363 (127.15.217.67:38919)
I20260810 07:15:39.867841 27566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "927a4a59afe843cbbee59c700a0e3083" candidate_uuid: "16935ff27a0e4a40b3f59ac3d0fbe95f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f135bd173b6425e81ea799289689be7"
I20260810 07:15:39.867913 27566 raft_consensus.cc:3037] T 927a4a59afe843cbbee59c700a0e3083 P 6f135bd173b6425e81ea799289689be7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:39.867869 27856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "927a4a59afe843cbbee59c700a0e3083" candidate_uuid: "16935ff27a0e4a40b3f59ac3d0fbe95f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4d0caa941be46e6afef35b541246363"
I20260810 07:15:39.867965 27856 raft_consensus.cc:3037] T 927a4a59afe843cbbee59c700a0e3083 P a4d0caa941be46e6afef35b541246363 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:39.868793 27566 raft_consensus.cc:2454] T 927a4a59afe843cbbee59c700a0e3083 P 6f135bd173b6425e81ea799289689be7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16935ff27a0e4a40b3f59ac3d0fbe95f in term 1.
I20260810 07:15:39.868798 27856 raft_consensus.cc:2454] T 927a4a59afe843cbbee59c700a0e3083 P a4d0caa941be46e6afef35b541246363 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16935ff27a0e4a40b3f59ac3d0fbe95f in term 1.
I20260810 07:15:39.868993 27643 leader_election.cc:302] T 927a4a59afe843cbbee59c700a0e3083 P 16935ff27a0e4a40b3f59ac3d0fbe95f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16935ff27a0e4a40b3f59ac3d0fbe95f, a4d0caa941be46e6afef35b541246363; no voters:
I20260810 07:15:39.869102 27939 raft_consensus.cc:2781] T 927a4a59afe843cbbee59c700a0e3083 P 16935ff27a0e4a40b3f59ac3d0fbe95f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:39.869278 27939 raft_consensus.cc:686] T 927a4a59afe843cbbee59c700a0e3083 P 16935ff27a0e4a40b3f59ac3d0fbe95f [term 1 LEADER]: Becoming Leader. State: Replica: 16935ff27a0e4a40b3f59ac3d0fbe95f, State: Running, Role: LEADER
I20260810 07:15:39.869385 27939 consensus_queue.cc:237] T 927a4a59afe843cbbee59c700a0e3083 P 16935ff27a0e4a40b3f59ac3d0fbe95f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16935ff27a0e4a40b3f59ac3d0fbe95f" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 38939 } } peers { permanent_uuid: "6f135bd173b6425e81ea799289689be7" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 35925 } } peers { permanent_uuid: "a4d0caa941be46e6afef35b541246363" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 38919 } }
I20260810 07:15:39.870410 27409 catalog_manager.cc:5495] T 927a4a59afe843cbbee59c700a0e3083 P 16935ff27a0e4a40b3f59ac3d0fbe95f reported cstate change: term changed from 0 to 1, leader changed from <none> to 16935ff27a0e4a40b3f59ac3d0fbe95f (127.15.217.66). New cstate: current_term: 1 leader_uuid: "16935ff27a0e4a40b3f59ac3d0fbe95f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16935ff27a0e4a40b3f59ac3d0fbe95f" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 38939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f135bd173b6425e81ea799289689be7" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 35925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4d0caa941be46e6afef35b541246363" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 38919 } health_report { overall_health: UNKNOWN } } }
I20260810 07:15:39.876540 27926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.874057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44844 (local address 127.15.217.66:38939)
0810 07:15:39.874133 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:39.874136 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:39.874144 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:15:39.874178 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:39.874179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:39.874198 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:39.874284 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:15:39.874396 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.874873 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.875659 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.875880 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.875974 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.876152 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:15:39.876267 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:15:39.876365 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:39.876385 (+ 20us) server_negotiation.cc:299] Negotiation successful
0810 07:15:39.876408 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 07:15:39.878549 27856 raft_consensus.cc:1261] T 927a4a59afe843cbbee59c700a0e3083 P a4d0caa941be46e6afef35b541246363 [term 1 FOLLOWER]: Refusing update from remote peer 16935ff27a0e4a40b3f59ac3d0fbe95f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:39.878569 27566 raft_consensus.cc:1261] T 927a4a59afe843cbbee59c700a0e3083 P 6f135bd173b6425e81ea799289689be7 [term 1 FOLLOWER]: Refusing update from remote peer 16935ff27a0e4a40b3f59ac3d0fbe95f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:39.878947 27939 consensus_queue.cc:1035] T 927a4a59afe843cbbee59c700a0e3083 P 16935ff27a0e4a40b3f59ac3d0fbe95f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4d0caa941be46e6afef35b541246363" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 38919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:39.879097 27942 consensus_queue.cc:1035] T 927a4a59afe843cbbee59c700a0e3083 P 16935ff27a0e4a40b3f59ac3d0fbe95f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f135bd173b6425e81ea799289689be7" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 35925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:39.881641 27945 mvcc.cc:205] Tried to move back new op lower bound from 7316873788939522048 to 7316873788906037248. Current Snapshot: MvccSnapshot[applied={T|T < 7316873788939522048}]
I20260810 07:15:39.882704 27946 mvcc.cc:205] Tried to move back new op lower bound from 7316873788939522048 to 7316873788906037248. Current Snapshot: MvccSnapshot[applied={T|T < 7316873788939522048}]
I20260810 07:15:39.884182 27411 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:39.884313 27411 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:39.885164 27411 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 669cb575a5d64842884330f7db404f6a: Sending DeleteTablet for 3 replicas of tablet 927a4a59afe843cbbee59c700a0e3083
I20260810 07:15:39.885478 27688 tablet_service.cc:1514] Processing DeleteTablet for tablet 927a4a59afe843cbbee59c700a0e3083 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:39 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:44828
I20260810 07:15:39.885614 27542 tablet_service.cc:1514] Processing DeleteTablet for tablet 927a4a59afe843cbbee59c700a0e3083 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:39 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:59546
I20260810 07:15:39.885653 27835 tablet_service.cc:1514] Processing DeleteTablet for tablet 927a4a59afe843cbbee59c700a0e3083 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:39 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:56914
I20260810 07:15:39.885887 27956 tablet_replica.cc:330] T 927a4a59afe843cbbee59c700a0e3083 P a4d0caa941be46e6afef35b541246363: stopping tablet replica
I20260810 07:15:39.885995 27956 raft_consensus.cc:2229] T 927a4a59afe843cbbee59c700a0e3083 P a4d0caa941be46e6afef35b541246363 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:39.886108 27956 raft_consensus.cc:2258] T 927a4a59afe843cbbee59c700a0e3083 P a4d0caa941be46e6afef35b541246363 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:39.886124 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 27476
I20260810 07:15:39.889580 27956 ts_tablet_manager.cc:1905] T 927a4a59afe843cbbee59c700a0e3083 P a4d0caa941be46e6afef35b541246363: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:39.891449 27956 ts_tablet_manager.cc:1918] T 927a4a59afe843cbbee59c700a0e3083 P a4d0caa941be46e6afef35b541246363: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:15:39.891549 27956 log.cc:1196] T 927a4a59afe843cbbee59c700a0e3083 P a4d0caa941be46e6afef35b541246363: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-2/wal/wals/927a4a59afe843cbbee59c700a0e3083
I20260810 07:15:39.891840 27956 ts_tablet_manager.cc:1939] T 927a4a59afe843cbbee59c700a0e3083 P a4d0caa941be46e6afef35b541246363: Deleting consensus metadata
I20260810 07:15:39.892319 27396 catalog_manager.cc:4841] TS a4d0caa941be46e6afef35b541246363 (127.15.217.67:38919): tablet 927a4a59afe843cbbee59c700a0e3083 (table test-table [id=818920f585fb4dffa696b469a73233c1]) successfully deleted
I20260810 07:15:39.896641 27954 tablet_replica.cc:330] T 927a4a59afe843cbbee59c700a0e3083 P 16935ff27a0e4a40b3f59ac3d0fbe95f: stopping tablet replica
I20260810 07:15:39.896797 27954 raft_consensus.cc:2229] T 927a4a59afe843cbbee59c700a0e3083 P 16935ff27a0e4a40b3f59ac3d0fbe95f [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:15:39.896929 27954 raft_consensus.cc:2258] T 927a4a59afe843cbbee59c700a0e3083 P 16935ff27a0e4a40b3f59ac3d0fbe95f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:39.897472 27954 ts_tablet_manager.cc:1905] T 927a4a59afe843cbbee59c700a0e3083 P 16935ff27a0e4a40b3f59ac3d0fbe95f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:39.899570 27954 ts_tablet_manager.cc:1918] T 927a4a59afe843cbbee59c700a0e3083 P 16935ff27a0e4a40b3f59ac3d0fbe95f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:15:39.899665 27954 log.cc:1196] T 927a4a59afe843cbbee59c700a0e3083 P 16935ff27a0e4a40b3f59ac3d0fbe95f: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346081759552-16229-0/minicluster-data/ts-1/wal/wals/927a4a59afe843cbbee59c700a0e3083
I20260810 07:15:39.900000 27954 ts_tablet_manager.cc:1939] T 927a4a59afe843cbbee59c700a0e3083 P 16935ff27a0e4a40b3f59ac3d0fbe95f: Deleting consensus metadata
I20260810 07:15:39.900553 27395 catalog_manager.cc:4841] TS 16935ff27a0e4a40b3f59ac3d0fbe95f (127.15.217.66:38939): tablet 927a4a59afe843cbbee59c700a0e3083 (table test-table [id=818920f585fb4dffa696b469a73233c1]) successfully deleted
W20260810 07:15:39.902436 27396 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.217.65:35925 (error 108)
I20260810 07:15:39.903134 27923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.902980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:35925 (local address 127.0.0.1:59558)
0810 07:15:39.903047 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:39.903096 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.65:35925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 07:15:39.903226 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 27623
W20260810 07:15:39.905624 27396 catalog_manager.cc:4584] TS 6f135bd173b6425e81ea799289689be7 (127.15.217.65:35925): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 927a4a59afe843cbbee59c700a0e3083: Network error: Client connection negotiation failed: client connection to 127.15.217.65:35925: connect: Connection refused (error 111)
I20260810 07:15:39.910735 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 27769
I20260810 07:15:39.920369 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 27374
2026-08-10T07:15:39Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2007 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:15:39 dist-test-slave-10pc krb5kdc[27961](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:15:39 dist-test-slave-10pc krb5kdc[27961](info): set up 2 sockets
Aug 10 07:15:39 dist-test-slave-10pc krb5kdc[27961](info): commencing operation
krb5kdc: starting...
W20260810 07:15:41.021189 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:15:41 dist-test-slave-10pc krb5kdc[27961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:15:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:15:41Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.15.217.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.15.217.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.15.217.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/rangeradmin_127.15.217.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.15.217.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/rangeradmin_127.15.217.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.15.217.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.15.217.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.15.217.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/rangerlookup_127.15.217.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.15.217.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/rangerlookup_127.15.217.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.217.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.15.217.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.217.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/HTTP_127.15.217.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.217.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/HTTP_127.15.217.84@KRBTEST.COM.keytab.
I20260810 07:15:41.205065 16229 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskvoWU8l/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/postgres -l logfile start
2026-08-10 07:15:43.718 UTC [27992] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 07:15:43.718 UTC [27992] LOG: listening on IPv4 address "127.15.217.84", port 34123
2026-08-10 07:15:43.722 UTC [27992] LOG: listening on Unix socket "/tmp/.s.PGSQL.34123"
2026-08-10 07:15:43.744 UTC [27995] LOG: database system was shut down at 2026-08-10 07:15:42 UTC
2026-08-10 07:15:43.748 UTC [27992] LOG: database system is ready to accept connections
I20260810 07:15:44.760690 16229 mini_postgres.cc:96] Postgres bound to 34123
2026-08-10 07:15:44.768 UTC [28003] FATAL: database "slave" does not exist
127.15.217.84:34123 - accepting connections
I20260810 07:15:44.769210 16229 mini_ranger.cc:161] Starting Ranger...
I20260810 07:15:44.785192 16229 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 07:15:45.254407 16229 mini_ranger.cc:87] Created ranger Postgres database
I20260810 07:15:45.254504 16229 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ranger-admin
2026-08-10 07:15:45,552 [I] DB FLAVOR :POSTGRES
2026-08-10 07:15:45,552 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 07:15:45,552 [I] Checking connection..
2026-08-10 07:15:45,553 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:34123/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 07:15:45,824 [I] Checking connection passed.
2026-08-10 07:15:45,824 [I] --------- Verifying version history table ---------
2026-08-10 07:15:45,824 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:34123/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 07:15:46,070 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 07:15:46,070 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:34123/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 07:15:46,332 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 07:15:46,332 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 07:15:46,332 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:34123/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 07:15:46.592 UTC [28078] WARNING: there is no transaction in progress
2026-08-10 07:15:46,600 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 07:15:46,600 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:34123/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 07:15:46,850 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 07:15:46,851 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 07:15:46,851 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:34123/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 07:15:47,116 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:34123/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 07:15:47,384 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:34123/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-10pc.c.gcp-upstream.internal','N') ;"
2026-08-10 07:15:47,630 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:34123/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 07:15:47,880 [I] Table x_portal_user does not exist in database ranger
2026-08-10 07:15:47,880 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:34123/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 07:15:48,139 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 07:15:48,140 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:34123/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 07:15:48,389 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:34123/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 07:15:48,646 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 07:15:48,647 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:34123/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 07:15:49.643 UTC [28197] WARNING: there is no transaction in progress
2026-08-10 07:15:49.659 UTC [28197] WARNING: there is no transaction in progress
2026-08-10 07:15:49.676 UTC [28197] WARNING: there is no transaction in progress
2026-08-10 07:15:49.692 UTC [28197] WARNING: there is no transaction in progress
2026-08-10 07:15:49.711 UTC [28197] WARNING: there is no transaction in progress
2026-08-10 07:15:49.733 UTC [28197] WARNING: there is no transaction in progress
2026-08-10 07:15:49.799 UTC [28197] WARNING: there is no transaction in progress
2026-08-10 07:15:49.806 UTC [28197] WARNING: there is no transaction in progress
2026-08-10 07:15:49.820 UTC [28197] WARNING: there is no transaction in progress
2026-08-10 07:15:49.830 UTC [28197] WARNING: there is no transaction in progress
2026-08-10 07:15:49.841 UTC [28197] WARNING: there is no transaction in progress
2026-08-10 07:15:49.856 UTC [28197] WARNING: there is no transaction in progress
2026-08-10 07:15:49.866 UTC [28197] WARNING: there is no transaction in progress
2026-08-10 07:15:49.872 UTC [28197] WARNING: there is no transaction in progress
2026-08-10 07:15:49.879 UTC [28197] WARNING: there is no transaction in progress
2026-08-10 07:15:49.886 UTC [28197] WARNING: there is no transaction in progress
2026-08-10 07:15:50,552 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 07:15:50,552 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:34123/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 07:15:50,809 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 07:15:50,810 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:34123/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 07:15:51,044 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 07:15:51,045 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:34123/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 07:15:51,315 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 07:15:51,316 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:34123/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 07:15:51,583 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:34123/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 07:15:51,828 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:34123/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-10pc.c.gcp-upstream.internal';"
2026-08-10 07:15:52,077 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 07:15:52,078 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:34123/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 07:15:52,340 [I] DB_PATCHES have already been applied
I20260810 07:15:52.347779 16229 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ranger-admin:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskvoWU8l/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 07:15:52.347847 16229 mini_ranger.cc:193] Using host: 127.15.217.84
I20260810 07:15:52.351898 16229 mini_ranger.cc:227] Ranger admin URL: http://127.15.217.84:50165
Aug 10, 2026 7:15:52 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 7:15:52 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 7:15:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 7:15:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 7:15:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 7:15:53 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 7:15:53 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 07:15:53.541173 16229 mini_ranger.cc:160] Time spent starting Ranger: real 8.772s user 0.000s sys 0.011s
Aug 10, 2026 7:16:13 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 7:16:13 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 10, 2026 7:16:13 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 7:16:13 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 7:16:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 7:16:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 7:16:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 7:16:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 7:16:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 7:16:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 7:16:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 7:16:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 7:16:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 7:16:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 7:16:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 7:16:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 7:16:14 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 7:16:14 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 07:16:15.468896 16229 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.15.217.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.15.217.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.15.217.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/rangerkms_127.15.217.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.15.217.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/rangerkms_127.15.217.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.217.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.15.217.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.217.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/HTTP_127.15.217.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.217.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/HTTP_127.15.217.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 07:16:15 dist-test-slave-10pc krb5kdc[27961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346175, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 07:16:15.560676 16229 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 07:16:15.571772 16229 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 07:16:16.442164 16229 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 07:16:16.442266 16229 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ranger-kms
I20260810 07:16:16.442288 16229 mini_ranger_kms.cc:227] Using postgres at 127.15.217.84:34123
2026-08-10 07:16:16,683 [I] DB FLAVOR :POSTGRES
2026-08-10 07:16:16,683 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 07:16:16,683 [I] Checking connection
2026-08-10 07:16:16,683 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:34123/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 07:16:16,929 [I] connection success
2026-08-10 07:16:16,929 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 07:16:16,929 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 07:16:16,929 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:34123/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 07:16:17,177 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 07:16:17,177 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 07:16:17,178 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 07:16:17,178 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:34123/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 07:16:17,456 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 07:16:17.951850 16229 mini_ranger_kms.cc:310] Created kms service
I20260810 07:16:18.128072 16229 mini_ranger_kms.cc:326] Created kudu user
I20260810 07:16:18.179865 16229 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 07:16:18 dist-test-slave-10pc krb5kdc[27961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346175, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.15.217.84@KRBTEST.COM
I20260810 07:16:18.641686 16229 mini_ranger_kms.cc:384] Added ranger policy
I20260810 07:16:18.641830 16229 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ranger-kms:/tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 07:16:18.641855 16229 mini_ranger_kms.cc:240] Using host: 127.15.217.84
I20260810 07:16:18.646287 16229 mini_ranger_kms.cc:276] Ranger KMS PID: 28402
I20260810 07:16:18.646436 16229 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.15.217.84:43047
Aug 10, 2026 7:16:19 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 7:16:19 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 7:16:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 7:16:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 7:16:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 7:16:20 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260810 07:16:21.011802 16229 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.451s user 0.005s sys 0.217s
I20260810 07:16:21.011933 16229 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 07:16:21.012027 16229 mini_ranger_kms.cc:401] 127.15.217.84:43047/kms/v1/keys
Aug 10 07:16:21 dist-test-slave-10pc krb5kdc[27961](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786346181, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.217.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 10 07:16:21 dist-test-slave-10pc krb5kdc[27961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346181, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.217.84@KRBTEST.COM for HTTP/127.15.217.84@KRBTEST.COM
Aug 10 07:16:21 dist-test-slave-10pc krb5kdc[27961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346181, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.217.84@KRBTEST.COM for HTTP/127.15.217.84@KRBTEST.COM
Aug 10 07:16:24 dist-test-slave-10pc krb5kdc[27961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.15.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
I20260810 07:16:24.297927 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:44607
--webserver_interface=127.15.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:44063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:44607
--ranger_config_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.217.84:43047/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:16:24.384789 28461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:16:24.385030 28461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:16:24.385054 28461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:16:24.386260 28461 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:16:24.386307 28461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:16:24.386322 28461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:16:24.386335 28461 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:16:24.386348 28461 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:16:24.388605 28461 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:44063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/master-0/wal
--ranger_kms_url=127.15.217.84:43047/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:44607
--ranger_config_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:44607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:16:24.388883 28461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:16:24.389158 28461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:16:24.394294 28469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:16:24.394310 28466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:16:24.394559 28461 server_base.cc:1034] running on GCE node
W20260810 07:16:24.394438 28472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:16:24.394986 28461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:16:24.395336 28461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:16:24.396528 28461 hybrid_clock.cc:648] HybridClock initialized: now 1786346184396512 us; error 28 us; skew 500 ppm
Aug 10 07:16:24 dist-test-slave-10pc krb5kdc[27961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346184, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:16:24.404016 28461 init.cc:377] Logged in from keytab as kudu/127.15.217.126@KRBTEST.COM (short username kudu)
I20260810 07:16:24.404603 28461 webserver.cc:460] Webserver started at http://127.15.217.126:43491/ using document root <none> and password file <none>
I20260810 07:16:24.404856 28461 fs_manager.cc:359] Metadata directory not provided
I20260810 07:16:24.404908 28461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:16:24.405061 28461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 07:16:24 dist-test-slave-10pc krb5kdc[27961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346184, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for HTTP/127.15.217.84@KRBTEST.COM
I20260810 07:16:24.461376 28461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "d556fcc65dce4e5785e1a46a93db8fcd"
format_stamp: "Formatted at 2026-08-10 07:16:24 on dist-test-slave-10pc"
server_key: "34b5b7a7ccd5f349eab8b737ac95c4b4"
server_key_iv: "4bbf04e14faa89241bf3aa781ec5aadb"
server_key_version: "kuduclusterkey@0"
I20260810 07:16:24.461840 28461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "d556fcc65dce4e5785e1a46a93db8fcd"
format_stamp: "Formatted at 2026-08-10 07:16:24 on dist-test-slave-10pc"
server_key: "34b5b7a7ccd5f349eab8b737ac95c4b4"
server_key_iv: "4bbf04e14faa89241bf3aa781ec5aadb"
server_key_version: "kuduclusterkey@0"
I20260810 07:16:24.463477 28461 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:16:24.480931 28480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:24.481473 28461 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:16:24.481607 28461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "d556fcc65dce4e5785e1a46a93db8fcd"
format_stamp: "Formatted at 2026-08-10 07:16:24 on dist-test-slave-10pc"
server_key: "34b5b7a7ccd5f349eab8b737ac95c4b4"
server_key_iv: "4bbf04e14faa89241bf3aa781ec5aadb"
server_key_version: "kuduclusterkey@0"
I20260810 07:16:24.481736 28461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:16:24.504611 28461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:16:24.506592 28461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:16:24.506732 28461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:16:24.511350 28461 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:44607
I20260810 07:16:24.511401 28535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:44607 every 8 connection(s)
I20260810 07:16:24.511829 28461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:16:24.512633 28536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:16:24.515035 28536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d556fcc65dce4e5785e1a46a93db8fcd: Bootstrap starting.
I20260810 07:16:24.515671 28536 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d556fcc65dce4e5785e1a46a93db8fcd: Neither blocks nor log segments found. Creating new log.
I20260810 07:16:24.515774 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 28461
I20260810 07:16:24.515901 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:16:24.516036 28536 log.cc:824] T 00000000000000000000000000000000 P d556fcc65dce4e5785e1a46a93db8fcd: Log is configured to *not* fsync() on all Append() calls
I20260810 07:16:24.516130 16229 external_mini_cluster.cc:1428] Setting key 1e9f9d8de6ffd963c0929d1d86bfee9e
I20260810 07:16:24.517234 28536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d556fcc65dce4e5785e1a46a93db8fcd: No bootstrap required, opened a new log
I20260810 07:16:24.519052 28536 raft_consensus.cc:348] T 00000000000000000000000000000000 P d556fcc65dce4e5785e1a46a93db8fcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d556fcc65dce4e5785e1a46a93db8fcd" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 44607 } }
I20260810 07:16:24.519212 28536 raft_consensus.cc:374] T 00000000000000000000000000000000 P d556fcc65dce4e5785e1a46a93db8fcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:16:24.519235 28536 raft_consensus.cc:729] T 00000000000000000000000000000000 P d556fcc65dce4e5785e1a46a93db8fcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d556fcc65dce4e5785e1a46a93db8fcd, State: Initialized, Role: FOLLOWER
I20260810 07:16:24.519372 28536 consensus_queue.cc:260] T 00000000000000000000000000000000 P d556fcc65dce4e5785e1a46a93db8fcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d556fcc65dce4e5785e1a46a93db8fcd" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 44607 } }
I20260810 07:16:24.519454 28536 raft_consensus.cc:388] T 00000000000000000000000000000000 P d556fcc65dce4e5785e1a46a93db8fcd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:16:24.519497 28536 raft_consensus.cc:482] T 00000000000000000000000000000000 P d556fcc65dce4e5785e1a46a93db8fcd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:16:24.519546 28536 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d556fcc65dce4e5785e1a46a93db8fcd [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:16:24.520172 28536 raft_consensus.cc:504] T 00000000000000000000000000000000 P d556fcc65dce4e5785e1a46a93db8fcd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d556fcc65dce4e5785e1a46a93db8fcd" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 44607 } }
I20260810 07:16:24.520284 28536 leader_election.cc:302] T 00000000000000000000000000000000 P d556fcc65dce4e5785e1a46a93db8fcd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d556fcc65dce4e5785e1a46a93db8fcd; no voters:
I20260810 07:16:24.520478 28536 leader_election.cc:288] T 00000000000000000000000000000000 P d556fcc65dce4e5785e1a46a93db8fcd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:16:24.520572 28542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d556fcc65dce4e5785e1a46a93db8fcd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:16:24.520730 28536 sys_catalog.cc:564] T 00000000000000000000000000000000 P d556fcc65dce4e5785e1a46a93db8fcd [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:16:24.520740 28542 raft_consensus.cc:686] T 00000000000000000000000000000000 P d556fcc65dce4e5785e1a46a93db8fcd [term 1 LEADER]: Becoming Leader. State: Replica: d556fcc65dce4e5785e1a46a93db8fcd, State: Running, Role: LEADER
I20260810 07:16:24.520959 28542 consensus_queue.cc:237] T 00000000000000000000000000000000 P d556fcc65dce4e5785e1a46a93db8fcd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d556fcc65dce4e5785e1a46a93db8fcd" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 44607 } }
I20260810 07:16:24.521663 28543 sys_catalog.cc:455] T 00000000000000000000000000000000 P d556fcc65dce4e5785e1a46a93db8fcd [sys.catalog]: SysCatalogTable state changed. Reason: New leader d556fcc65dce4e5785e1a46a93db8fcd. Latest consensus state: current_term: 1 leader_uuid: "d556fcc65dce4e5785e1a46a93db8fcd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d556fcc65dce4e5785e1a46a93db8fcd" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 44607 } } }
I20260810 07:16:24.521756 28543 sys_catalog.cc:458] T 00000000000000000000000000000000 P d556fcc65dce4e5785e1a46a93db8fcd [sys.catalog]: This master's current role is: LEADER
I20260810 07:16:24.521994 28544 sys_catalog.cc:455] T 00000000000000000000000000000000 P d556fcc65dce4e5785e1a46a93db8fcd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d556fcc65dce4e5785e1a46a93db8fcd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d556fcc65dce4e5785e1a46a93db8fcd" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 44607 } } }
I20260810 07:16:24.522110 28544 sys_catalog.cc:458] T 00000000000000000000000000000000 P d556fcc65dce4e5785e1a46a93db8fcd [sys.catalog]: This master's current role is: LEADER
I20260810 07:16:24.522277 28552 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:16:24.522671 28536 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 07:16:24.523020 28552 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 07:16:24 dist-test-slave-10pc krb5kdc[27961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:16:24.528641 28552 catalog_manager.cc:1347] Generated new cluster ID: 535413d5057a4f3ea0dea9f5ddcd6612
I20260810 07:16:24.528767 28552 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:16:24.532166 28541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:24.516760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36454 (local address 127.15.217.126:44607)
0810 07:16:24.516948 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:24.516957 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:24.516986 (+ 29us) server_negotiation.cc:407] Connection header received
0810 07:16:24.517482 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:24.517490 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:24.517542 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:24.517682 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:16:24.518490 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:24.519942 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:24.521090 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:24.521756 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:24.521895 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:24.525649 (+ 3754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:16:24.525719 (+ 70us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:16:24.525719 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:16:24.525858 (+ 139us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:16:24.529620 (+ 3762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:16:24.530102 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:16:24.530108 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:16:24.530108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:16:24.530227 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:16:24.530505 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:16:24.530506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:16:24.530506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:16:24.530863 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:16:24.531674 (+ 811us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:24.531754 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 07:16:24.531919 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":90,"threads_started":1}
I20260810 07:16:24.561189 28552 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:16:24.561806 28552 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:16:24.565546 28552 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d556fcc65dce4e5785e1a46a93db8fcd: Generated new TSK 0
I20260810 07:16:24.565760 28552 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab.
Entry for principal kudu/127.15.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab.
I20260810 07:16:25.256129 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.217.84:43047/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.65:0
--local_ip_for_outbound_sockets=127.15.217.65
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:44607
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:44063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:16:25.361765 28586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:16:25.362047 28586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:16:25.362092 28586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:16:25.363721 28586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:16:25.363801 28586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:16:25.363850 28586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.65
I20260810 07:16:25.365811 28586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:44063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.15.217.84:43047/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:44607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.65
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:16:25.366125 28586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:16:25.366405 28586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:16:25.371973 28592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:16:25.372156 28594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:16:25.372330 28591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:16:25.373023 28586 server_base.cc:1034] running on GCE node
I20260810 07:16:25.373243 28586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:16:25.373507 28586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:16:25.374703 28586 hybrid_clock.cc:648] HybridClock initialized: now 1786346185374688 us; error 32 us; skew 500 ppm
Aug 10 07:16:25 dist-test-slave-10pc krb5kdc[27961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346185, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:16:25.377696 28586 init.cc:377] Logged in from keytab as kudu/127.15.217.65@KRBTEST.COM (short username kudu)
I20260810 07:16:25.378259 28586 webserver.cc:460] Webserver started at http://127.15.217.65:41139/ using document root <none> and password file <none>
I20260810 07:16:25.378536 28586 fs_manager.cc:359] Metadata directory not provided
I20260810 07:16:25.378616 28586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:16:25.378800 28586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 07:16:25 dist-test-slave-10pc krb5kdc[27961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346185, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for HTTP/127.15.217.84@KRBTEST.COM
I20260810 07:16:25.394162 28586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-0/data/instance:
uuid: "a68b586e8d3d42b3a2cb32b080d6291d"
format_stamp: "Formatted at 2026-08-10 07:16:25 on dist-test-slave-10pc"
server_key: "c21b2ea651031e902ba233836d325ed5"
server_key_iv: "24783ec6cb1e7101c843b8e3d991fe96"
server_key_version: "kuduclusterkey@0"
I20260810 07:16:25.394799 28586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance:
uuid: "a68b586e8d3d42b3a2cb32b080d6291d"
format_stamp: "Formatted at 2026-08-10 07:16:25 on dist-test-slave-10pc"
server_key: "c21b2ea651031e902ba233836d325ed5"
server_key_iv: "24783ec6cb1e7101c843b8e3d991fe96"
server_key_version: "kuduclusterkey@0"
I20260810 07:16:25.397578 28586 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:16:25.414424 28601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:25.414690 28586 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260810 07:16:25.414805 28586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-0/wal
uuid: "a68b586e8d3d42b3a2cb32b080d6291d"
format_stamp: "Formatted at 2026-08-10 07:16:25 on dist-test-slave-10pc"
server_key: "c21b2ea651031e902ba233836d325ed5"
server_key_iv: "24783ec6cb1e7101c843b8e3d991fe96"
server_key_version: "kuduclusterkey@0"
I20260810 07:16:25.414935 28586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:16:25.436317 28586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:16:25.439049 28586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:16:25.439272 28586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:16:25.439563 28586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:16:25.439980 28586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:16:25.440052 28586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:25.440100 28586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:16:25.440133 28586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:25.447073 28586 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.65:39139
I20260810 07:16:25.447630 28586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
I20260810 07:16:25.455547 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 28586
I20260810 07:16:25.455556 28732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.65:39139 every 8 connection(s)
I20260810 07:16:25.455685 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance
I20260810 07:16:25.455921 16229 external_mini_cluster.cc:1428] Setting key e831048c7b2934ba018819a9471874ff
Aug 10 07:16:25 dist-test-slave-10pc krb5kdc[27961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346185, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:16:25.468336 28736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:25.456604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:52863 (local address 127.15.217.126:44607)
0810 07:16:25.458072 (+ 1468us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:25.458075 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:25.458083 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:16:25.458110 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:25.458111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:25.458124 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:25.458195 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:16:25.459080 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:25.459931 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:25.461595 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:25.461946 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:25.462032 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:25.464539 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:16:25.464541 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:16:25.464541 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:16:25.464554 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:16:25.466621 (+ 2067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:16:25.467156 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:16:25.467157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:16:25.467158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:16:25.467228 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:16:25.467528 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:16:25.467530 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:16:25.467530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:16:25.467684 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:16:25.467749 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:25.468111 (+ 362us) server_negotiation.cc:299] Negotiation successful
0810 07:16:25.468211 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1437,"thread_start_us":103,"threads_started":1}
I20260810 07:16:25.469187 28735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:25.456567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:44607 (local address 127.15.217.65:52863)
0810 07:16:25.457042 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0810 07:16:25.457102 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 07:16:25.457791 (+ 689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:16:25.458349 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:16:25.458354 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:16:25.458619 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:16:25.458943 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:25.458951 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:25.460089 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:25.460091 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:25.461466 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:25.461469 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:25.462070 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:25.462072 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:25.462233 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:25.462940 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:16:25.462952 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:16:25.464375 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:16:25.466723 (+ 2348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:16:25.466731 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:16:25.466733 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:16:25.467062 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:16:25.467294 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:16:25.467295 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:16:25.467295 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:16:25.467446 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:16:25.467780 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:16:25.467788 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:16:25.467948 (+ 160us) client_negotiation.cc:769] Sending connection context
0810 07:16:25.468070 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 07:16:25.468234 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":107,"threads_started":1}
WARNING: no policy specified for kudu/127.15.217.66@KRBTEST.COM; defaulting to no policy
I20260810 07:16:25.470279 28733 heartbeater.cc:347] Connected to a master server at 127.15.217.126:44607
I20260810 07:16:25.470464 28733 heartbeater.cc:464] Registering TS with master...
I20260810 07:16:25.470844 28733 heartbeater.cc:511] Master 127.15.217.126:44607 requested a full tablet report, sending...
I20260810 07:16:25.471720 28497 ts_manager.cc:194] Registered new tserver with Master: a68b586e8d3d42b3a2cb32b080d6291d (127.15.217.65:39139)
I20260810 07:16:25.472697 28497 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.65@KRBTEST.COM'} at 127.15.217.65:52863
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab.
Entry for principal kudu/127.15.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab.
I20260810 07:16:25.499358 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.217.84:43047/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.66:0
--local_ip_for_outbound_sockets=127.15.217.66
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:44607
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:44063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 07:16:25 dist-test-slave-10pc krb5kdc[27961](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786346185, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 07:16:25.630832 28739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:16:25.631217 28739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:16:25.631306 28739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:16:25.633286 28739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:16:25.633435 28739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:16:25.633519 28739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.66
I20260810 07:16:25.635974 28739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:44063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.15.217.84:43047/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:44607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.66
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:16:25.636528 28739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:16:25.636857 28739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:16:25.643826 28746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:16:25.644097 28749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:16:25.644387 28747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:16:25.645692 28739 server_base.cc:1034] running on GCE node
I20260810 07:16:25.645929 28739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:16:25.646236 28739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:16:25.647439 28739 hybrid_clock.cc:648] HybridClock initialized: now 1786346185647404 us; error 48 us; skew 500 ppm
Aug 10 07:16:25 dist-test-slave-10pc krb5kdc[27961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346185, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:16:25.651087 28739 init.cc:377] Logged in from keytab as kudu/127.15.217.66@KRBTEST.COM (short username kudu)
I20260810 07:16:25.651698 28739 webserver.cc:460] Webserver started at http://127.15.217.66:34105/ using document root <none> and password file <none>
I20260810 07:16:25.652001 28739 fs_manager.cc:359] Metadata directory not provided
I20260810 07:16:25.652072 28739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:16:25.652289 28739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 07:16:25 dist-test-slave-10pc krb5kdc[27961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346185, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for HTTP/127.15.217.84@KRBTEST.COM
I20260810 07:16:25.666806 28739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-1/data/instance:
uuid: "a34e63934b9d4bee8f54d2db3554a5eb"
format_stamp: "Formatted at 2026-08-10 07:16:25 on dist-test-slave-10pc"
server_key: "d8d0cddf410b22af948b3da706aac91e"
server_key_iv: "2f6044f2d210df8947b666b11ffbf165"
server_key_version: "kuduclusterkey@0"
I20260810 07:16:25.673003 28739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance:
uuid: "a34e63934b9d4bee8f54d2db3554a5eb"
format_stamp: "Formatted at 2026-08-10 07:16:25 on dist-test-slave-10pc"
server_key: "d8d0cddf410b22af948b3da706aac91e"
server_key_iv: "2f6044f2d210df8947b666b11ffbf165"
server_key_version: "kuduclusterkey@0"
I20260810 07:16:25.676568 28739 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 07:16:25.693778 28758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:25.700663 28739 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.000s sys 0.001s
I20260810 07:16:25.700830 28739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-1/wal
uuid: "a34e63934b9d4bee8f54d2db3554a5eb"
format_stamp: "Formatted at 2026-08-10 07:16:25 on dist-test-slave-10pc"
server_key: "d8d0cddf410b22af948b3da706aac91e"
server_key_iv: "2f6044f2d210df8947b666b11ffbf165"
server_key_version: "kuduclusterkey@0"
I20260810 07:16:25.700958 28739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:16:25.719405 28739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:16:25.722221 28739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:16:25.722388 28739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:16:25.722641 28739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:16:25.723055 28739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:16:25.723110 28739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:25.723143 28739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:16:25.723170 28739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:25.730278 28739 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.66:42725
I20260810 07:16:25.730764 28739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
I20260810 07:16:25.732404 28884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.66:42725 every 8 connection(s)
I20260810 07:16:25.736797 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 28739
I20260810 07:16:25.736927 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance
I20260810 07:16:25.737155 16229 external_mini_cluster.cc:1428] Setting key f2fae7f56b210885bea1178d2c80e334
Aug 10 07:16:25 dist-test-slave-10pc krb5kdc[27961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346185, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:16:25.745062 28736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:25.733574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:48659 (local address 127.15.217.126:44607)
0810 07:16:25.733624 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:25.733626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:25.733639 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:16:25.734319 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:25.734321 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:25.734338 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:25.734370 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:16:25.735273 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:25.736075 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:25.738122 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:25.738448 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:25.738491 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:25.741370 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:16:25.741372 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:16:25.741372 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:16:25.741381 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:16:25.743217 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:16:25.743858 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:16:25.743859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:16:25.743859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:16:25.743923 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:16:25.744201 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:16:25.744201 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:16:25.744202 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:16:25.744349 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:16:25.744392 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:25.744828 (+ 436us) server_negotiation.cc:299] Negotiation successful
0810 07:16:25.744953 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 07:16:25.745682 28887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:25.733220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:44607 (local address 127.15.217.66:48659)
0810 07:16:25.733459 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0810 07:16:25.733519 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 07:16:25.734176 (+ 657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:16:25.734571 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:16:25.734575 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:16:25.734857 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:16:25.735151 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:25.735158 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:25.736206 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:25.736207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:25.738011 (+ 1804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:25.738015 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:25.738590 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:25.738591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:25.738755 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:25.739402 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:16:25.739414 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:16:25.741230 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:16:25.743323 (+ 2093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:16:25.743333 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:16:25.743336 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:16:25.743763 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:16:25.743991 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:16:25.743991 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:16:25.743992 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:16:25.744129 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:16:25.744438 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:16:25.744445 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:16:25.744664 (+ 219us) client_negotiation.cc:769] Sending connection context
0810 07:16:25.745220 (+ 556us) client_negotiation.cc:240] Negotiation successful
0810 07:16:25.745410 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":119,"threads_started":1}
I20260810 07:16:25.746150 28885 heartbeater.cc:347] Connected to a master server at 127.15.217.126:44607
I20260810 07:16:25.746263 28885 heartbeater.cc:464] Registering TS with master...
I20260810 07:16:25.746491 28885 heartbeater.cc:511] Master 127.15.217.126:44607 requested a full tablet report, sending...
I20260810 07:16:25.746883 28497 ts_manager.cc:194] Registered new tserver with Master: a34e63934b9d4bee8f54d2db3554a5eb (127.15.217.66:42725)
I20260810 07:16:25.747411 28497 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.66@KRBTEST.COM'} at 127.15.217.66:48659
WARNING: no policy specified for kudu/127.15.217.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab.
Entry for principal kudu/127.15.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab.
I20260810 07:16:25.791620 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.217.84:43047/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.67:0
--local_ip_for_outbound_sockets=127.15.217.67
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:44607
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:44063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:16:25.941891 28890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:16:25.942145 28890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:16:25.942194 28890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:16:25.955178 28890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:16:25.955278 28890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:16:25.955327 28890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.67
I20260810 07:16:25.958091 28890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:44063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.15.217.84:43047/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:44607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.67
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:16:25.958401 28890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:16:25.958701 28890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:16:25.965524 28897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:16:25.965543 28899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:16:25.965786 28895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:16:25.966888 28890 server_base.cc:1034] running on GCE node
I20260810 07:16:25.967223 28890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:16:25.967527 28890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:16:25.968708 28890 hybrid_clock.cc:648] HybridClock initialized: now 1786346185968683 us; error 36 us; skew 500 ppm
Aug 10 07:16:25 dist-test-slave-10pc krb5kdc[27961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346185, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:16:25.972281 28890 init.cc:377] Logged in from keytab as kudu/127.15.217.67@KRBTEST.COM (short username kudu)
I20260810 07:16:25.972879 28890 webserver.cc:460] Webserver started at http://127.15.217.67:38253/ using document root <none> and password file <none>
I20260810 07:16:25.973181 28890 fs_manager.cc:359] Metadata directory not provided
I20260810 07:16:25.973255 28890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:16:25.973412 28890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 07:16:25 dist-test-slave-10pc krb5kdc[27961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346185, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for HTTP/127.15.217.84@KRBTEST.COM
I20260810 07:16:25.994187 28890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-2/data/instance:
uuid: "dac9dd9a408f4699aadfb69882197709"
format_stamp: "Formatted at 2026-08-10 07:16:25 on dist-test-slave-10pc"
server_key: "3b49289e6a6272dc70d3fd806157496c"
server_key_iv: "28088d17741c2cfc8fdae30514aa08ea"
server_key_version: "kuduclusterkey@0"
I20260810 07:16:25.994685 28890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance:
uuid: "dac9dd9a408f4699aadfb69882197709"
format_stamp: "Formatted at 2026-08-10 07:16:25 on dist-test-slave-10pc"
server_key: "3b49289e6a6272dc70d3fd806157496c"
server_key_iv: "28088d17741c2cfc8fdae30514aa08ea"
server_key_version: "kuduclusterkey@0"
I20260810 07:16:25.996507 28890 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260810 07:16:26.016698 28906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:26.016908 28890 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:16:26.017002 28890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-2/wal
uuid: "dac9dd9a408f4699aadfb69882197709"
format_stamp: "Formatted at 2026-08-10 07:16:25 on dist-test-slave-10pc"
server_key: "3b49289e6a6272dc70d3fd806157496c"
server_key_iv: "28088d17741c2cfc8fdae30514aa08ea"
server_key_version: "kuduclusterkey@0"
I20260810 07:16:26.017122 28890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:16:26.032295 28890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:16:26.034936 28890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:16:26.035094 28890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:16:26.035301 28890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:16:26.035679 28890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:16:26.035727 28890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:26.035758 28890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:16:26.035780 28890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:26.042512 28890 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.67:35087
I20260810 07:16:26.042977 28890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
I20260810 07:16:26.044498 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 28890
I20260810 07:16:26.044623 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance
I20260810 07:16:26.044854 16229 external_mini_cluster.cc:1428] Setting key 116302b4404858f65af9d7aa4b7d6346
I20260810 07:16:26.047091 29034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.67:35087 every 8 connection(s)
Aug 10 07:16:26 dist-test-slave-10pc krb5kdc[27961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346185, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:16:26.059819 28736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:26.048170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:44875 (local address 127.15.217.126:44607)
0810 07:16:26.048217 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:26.048219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:26.048708 (+ 489us) server_negotiation.cc:407] Connection header received
0810 07:16:26.049421 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:26.049422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:26.049434 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:26.049464 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:16:26.050331 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:26.051082 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:26.052711 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:26.052994 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:26.053037 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:26.055814 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:16:26.055815 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:16:26.055816 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:16:26.055824 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:16:26.058226 (+ 2402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:16:26.058736 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:16:26.058738 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:16:26.058738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:16:26.058799 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:16:26.059068 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:16:26.059069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:16:26.059070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:16:26.059199 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:16:26.059242 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:26.059625 (+ 383us) server_negotiation.cc:299] Negotiation successful
0810 07:16:26.059722 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 07:16:26.060654 29037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:26.047815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:44607 (local address 127.15.217.67:44875)
0810 07:16:26.048509 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0810 07:16:26.048558 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 07:16:26.049292 (+ 734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:16:26.049668 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:16:26.049672 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:16:26.049927 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:16:26.050218 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:26.050225 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:26.051210 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:26.051211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:26.052610 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:26.052613 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:26.053134 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:26.053135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:26.053276 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:26.053956 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:16:26.053968 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:16:26.055701 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:16:26.058326 (+ 2625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:16:26.058333 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:16:26.058336 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:16:26.058646 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:16:26.058869 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:16:26.058869 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:16:26.058870 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:16:26.059000 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:16:26.059289 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:16:26.059295 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:16:26.059465 (+ 170us) client_negotiation.cc:769] Sending connection context
0810 07:16:26.060192 (+ 727us) client_negotiation.cc:240] Negotiation successful
0810 07:16:26.060367 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"thread_start_us":126,"threads_started":1}
I20260810 07:16:26.061347 29035 heartbeater.cc:347] Connected to a master server at 127.15.217.126:44607
I20260810 07:16:26.061446 29035 heartbeater.cc:464] Registering TS with master...
I20260810 07:16:26.061654 29035 heartbeater.cc:511] Master 127.15.217.126:44607 requested a full tablet report, sending...
I20260810 07:16:26.062060 28497 ts_manager.cc:194] Registered new tserver with Master: dac9dd9a408f4699aadfb69882197709 (127.15.217.67:35087)
I20260810 07:16:26.062528 28497 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.67@KRBTEST.COM'} at 127.15.217.67:44875
I20260810 07:16:26.069082 16229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:16:26.081856 28736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:26.072076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36468 (local address 127.15.217.126:44607)
0810 07:16:26.072123 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:26.072125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:26.072524 (+ 399us) server_negotiation.cc:407] Connection header received
0810 07:16:26.072639 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:26.072640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:26.072652 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:26.072680 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:16:26.073474 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:26.074225 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:26.075531 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:26.075833 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:26.075874 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:26.077510 (+ 1636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:16:26.077512 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:16:26.077512 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:16:26.077520 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:16:26.080543 (+ 3023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:16:26.080974 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:16:26.080976 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:16:26.080976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:16:26.081037 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:16:26.081230 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:16:26.081231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:16:26.081231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:16:26.081357 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:16:26.081433 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:26.081671 (+ 238us) server_negotiation.cc:299] Negotiation successful
0810 07:16:26.081754 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 07:16:26.084224 28497 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36468:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:16:26.086695 28497 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:16:26.104295 29051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:26.089833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:39139 (local address 127.0.0.1:41236)
0810 07:16:26.091422 (+ 1589us) negotiation.cc:107] Waiting for socket to connect
0810 07:16:26.091470 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 07:16:26.091534 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:16:26.092092 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:16:26.092095 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:16:26.092098 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:16:26.092195 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:26.092198 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:26.097065 (+ 4867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:26.097066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:26.100710 (+ 3644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:26.100713 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:26.103892 (+ 3179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:26.103893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:26.104008 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:26.104028 (+ 20us) client_negotiation.cc:769] Sending connection context
0810 07:16:26.104111 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 07:16:26.104196 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1554,"thread_start_us":100,"threads_started":1}
I20260810 07:16:26.104732 29049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:26.090441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41236 (local address 127.15.217.65:39139)
0810 07:16:26.091889 (+ 1448us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:26.091893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:26.091903 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:16:26.091937 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:26.091939 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:26.091956 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:26.092021 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:16:26.092764 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:26.096954 (+ 4190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:26.102951 (+ 5997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:26.103704 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:26.103783 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:26.103849 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:26.104607 (+ 758us) server_negotiation.cc:299] Negotiation successful
0810 07:16:26.104661 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1418,"thread_start_us":59,"threads_started":1}
I20260810 07:16:26.105721 28657 tablet_service.cc:1467] Processing CreateTablet for tablet 3fdb36db7295465d93c03160c8085a31 (DEFAULT_TABLE table=test-table [id=4f93d0e446ab418daedbe5584682e411]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:16:26.106187 28657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fdb36db7295465d93c03160c8085a31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:16:26.107676 29048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:26.090105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:35087 (local address 127.0.0.1:40306)
0810 07:16:26.091740 (+ 1635us) negotiation.cc:107] Waiting for socket to connect
0810 07:16:26.091749 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 07:16:26.091792 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:16:26.092622 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:16:26.092623 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:16:26.092626 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:16:26.092676 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:26.092678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:26.094427 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:26.094428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:26.101355 (+ 6927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:26.101359 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:26.107401 (+ 6042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:26.107402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:26.107472 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:26.107477 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:16:26.107534 (+ 57us) client_negotiation.cc:240] Negotiation successful
0810 07:16:26.107579 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1620,"thread_start_us":95,"threads_started":1}
I20260810 07:16:26.108023 29047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:26.090835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:42725 (local address 127.0.0.1:56898)
0810 07:16:26.095696 (+ 4861us) negotiation.cc:107] Waiting for socket to connect
0810 07:16:26.095706 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:16:26.095762 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:16:26.098511 (+ 2749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:16:26.098512 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:16:26.098514 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:16:26.098560 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:26.098561 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:26.099944 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:26.099945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:26.101419 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:26.101421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:26.107784 (+ 6363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:26.107785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:26.107846 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:26.107850 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:16:26.107898 (+ 48us) client_negotiation.cc:240] Negotiation successful
0810 07:16:26.107943 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4842,"thread_start_us":62,"threads_started":1}
I20260810 07:16:26.108283 29050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:26.090604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40306 (local address 127.15.217.67:35087)
0810 07:16:26.092330 (+ 1726us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:26.092335 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:26.092344 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:16:26.092383 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:26.092386 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:26.092401 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:26.092540 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:16:26.093291 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:26.094319 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:26.106395 (+ 12076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:26.107181 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:26.107265 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:26.107356 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:26.108140 (+ 784us) server_negotiation.cc:299] Negotiation successful
0810 07:16:26.108190 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1687,"thread_start_us":85,"threads_started":1}
I20260810 07:16:26.109380 28957 tablet_service.cc:1467] Processing CreateTablet for tablet 3fdb36db7295465d93c03160c8085a31 (DEFAULT_TABLE table=test-table [id=4f93d0e446ab418daedbe5584682e411]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:16:26.109762 28957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fdb36db7295465d93c03160c8085a31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:16:26.117053 29052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:26.091120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56898 (local address 127.15.217.66:42725)
0810 07:16:26.091688 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:26.091694 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:26.095826 (+ 4132us) server_negotiation.cc:407] Connection header received
0810 07:16:26.095877 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:26.095880 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:26.095896 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:26.095969 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:16:26.098653 (+ 2684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:26.099834 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:26.101532 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:26.102732 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:26.102813 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:26.102903 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:26.116734 (+ 13831us) server_negotiation.cc:299] Negotiation successful
0810 07:16:26.116958 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":90,"threads_started":1}
I20260810 07:16:26.117954 28793 tablet_service.cc:1467] Processing CreateTablet for tablet 3fdb36db7295465d93c03160c8085a31 (DEFAULT_TABLE table=test-table [id=4f93d0e446ab418daedbe5584682e411]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:16:26.118315 28793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fdb36db7295465d93c03160c8085a31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:16:26.124624 29053 tablet_bootstrap.cc:492] T 3fdb36db7295465d93c03160c8085a31 P a68b586e8d3d42b3a2cb32b080d6291d: Bootstrap starting.
I20260810 07:16:26.125499 29053 tablet_bootstrap.cc:654] T 3fdb36db7295465d93c03160c8085a31 P a68b586e8d3d42b3a2cb32b080d6291d: Neither blocks nor log segments found. Creating new log.
I20260810 07:16:26.125900 29053 log.cc:824] T 3fdb36db7295465d93c03160c8085a31 P a68b586e8d3d42b3a2cb32b080d6291d: Log is configured to *not* fsync() on all Append() calls
I20260810 07:16:26.126343 29054 tablet_bootstrap.cc:492] T 3fdb36db7295465d93c03160c8085a31 P dac9dd9a408f4699aadfb69882197709: Bootstrap starting.
I20260810 07:16:26.127367 29054 tablet_bootstrap.cc:654] T 3fdb36db7295465d93c03160c8085a31 P dac9dd9a408f4699aadfb69882197709: Neither blocks nor log segments found. Creating new log.
I20260810 07:16:26.127743 29054 log.cc:824] T 3fdb36db7295465d93c03160c8085a31 P dac9dd9a408f4699aadfb69882197709: Log is configured to *not* fsync() on all Append() calls
I20260810 07:16:26.128615 29053 tablet_bootstrap.cc:492] T 3fdb36db7295465d93c03160c8085a31 P a68b586e8d3d42b3a2cb32b080d6291d: No bootstrap required, opened a new log
I20260810 07:16:26.128733 29053 ts_tablet_manager.cc:1397] T 3fdb36db7295465d93c03160c8085a31 P a68b586e8d3d42b3a2cb32b080d6291d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260810 07:16:26.131597 29053 raft_consensus.cc:348] T 3fdb36db7295465d93c03160c8085a31 P a68b586e8d3d42b3a2cb32b080d6291d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a34e63934b9d4bee8f54d2db3554a5eb" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 42725 } } peers { permanent_uuid: "a68b586e8d3d42b3a2cb32b080d6291d" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 39139 } } peers { permanent_uuid: "dac9dd9a408f4699aadfb69882197709" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 35087 } }
I20260810 07:16:26.131922 29053 raft_consensus.cc:374] T 3fdb36db7295465d93c03160c8085a31 P a68b586e8d3d42b3a2cb32b080d6291d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:16:26.132016 29053 raft_consensus.cc:729] T 3fdb36db7295465d93c03160c8085a31 P a68b586e8d3d42b3a2cb32b080d6291d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a68b586e8d3d42b3a2cb32b080d6291d, State: Initialized, Role: FOLLOWER
I20260810 07:16:26.132234 29053 consensus_queue.cc:260] T 3fdb36db7295465d93c03160c8085a31 P a68b586e8d3d42b3a2cb32b080d6291d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a34e63934b9d4bee8f54d2db3554a5eb" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 42725 } } peers { permanent_uuid: "a68b586e8d3d42b3a2cb32b080d6291d" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 39139 } } peers { permanent_uuid: "dac9dd9a408f4699aadfb69882197709" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 35087 } }
I20260810 07:16:26.132869 29053 ts_tablet_manager.cc:1428] T 3fdb36db7295465d93c03160c8085a31 P a68b586e8d3d42b3a2cb32b080d6291d: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260810 07:16:26.133065 28733 heartbeater.cc:503] Master 127.15.217.126:44607 was elected leader, sending a full tablet report...
I20260810 07:16:26.133694 29055 tablet_bootstrap.cc:492] T 3fdb36db7295465d93c03160c8085a31 P a34e63934b9d4bee8f54d2db3554a5eb: Bootstrap starting.
I20260810 07:16:26.134795 29055 tablet_bootstrap.cc:654] T 3fdb36db7295465d93c03160c8085a31 P a34e63934b9d4bee8f54d2db3554a5eb: Neither blocks nor log segments found. Creating new log.
I20260810 07:16:26.135195 29055 log.cc:824] T 3fdb36db7295465d93c03160c8085a31 P a34e63934b9d4bee8f54d2db3554a5eb: Log is configured to *not* fsync() on all Append() calls
I20260810 07:16:26.145316 29054 tablet_bootstrap.cc:492] T 3fdb36db7295465d93c03160c8085a31 P dac9dd9a408f4699aadfb69882197709: No bootstrap required, opened a new log
I20260810 07:16:26.145507 29054 ts_tablet_manager.cc:1397] T 3fdb36db7295465d93c03160c8085a31 P dac9dd9a408f4699aadfb69882197709: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20260810 07:16:26.147539 29054 raft_consensus.cc:348] T 3fdb36db7295465d93c03160c8085a31 P dac9dd9a408f4699aadfb69882197709 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a34e63934b9d4bee8f54d2db3554a5eb" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 42725 } } peers { permanent_uuid: "a68b586e8d3d42b3a2cb32b080d6291d" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 39139 } } peers { permanent_uuid: "dac9dd9a408f4699aadfb69882197709" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 35087 } }
I20260810 07:16:26.147814 29054 raft_consensus.cc:374] T 3fdb36db7295465d93c03160c8085a31 P dac9dd9a408f4699aadfb69882197709 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:16:26.147891 29054 raft_consensus.cc:729] T 3fdb36db7295465d93c03160c8085a31 P dac9dd9a408f4699aadfb69882197709 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dac9dd9a408f4699aadfb69882197709, State: Initialized, Role: FOLLOWER
I20260810 07:16:26.148092 29054 consensus_queue.cc:260] T 3fdb36db7295465d93c03160c8085a31 P dac9dd9a408f4699aadfb69882197709 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a34e63934b9d4bee8f54d2db3554a5eb" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 42725 } } peers { permanent_uuid: "a68b586e8d3d42b3a2cb32b080d6291d" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 39139 } } peers { permanent_uuid: "dac9dd9a408f4699aadfb69882197709" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 35087 } }
I20260810 07:16:26.148484 29054 ts_tablet_manager.cc:1428] T 3fdb36db7295465d93c03160c8085a31 P dac9dd9a408f4699aadfb69882197709: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:16:26.148648 29035 heartbeater.cc:503] Master 127.15.217.126:44607 was elected leader, sending a full tablet report...
I20260810 07:16:26.149729 29055 tablet_bootstrap.cc:492] T 3fdb36db7295465d93c03160c8085a31 P a34e63934b9d4bee8f54d2db3554a5eb: No bootstrap required, opened a new log
I20260810 07:16:26.149869 29055 ts_tablet_manager.cc:1397] T 3fdb36db7295465d93c03160c8085a31 P a34e63934b9d4bee8f54d2db3554a5eb: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
Aug 10 07:16:26 dist-test-slave-10pc krb5kdc[27961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346185, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for HTTP/127.15.217.84@KRBTEST.COM
I20260810 07:16:26.151989 29055 raft_consensus.cc:348] T 3fdb36db7295465d93c03160c8085a31 P a34e63934b9d4bee8f54d2db3554a5eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a34e63934b9d4bee8f54d2db3554a5eb" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 42725 } } peers { permanent_uuid: "a68b586e8d3d42b3a2cb32b080d6291d" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 39139 } } peers { permanent_uuid: "dac9dd9a408f4699aadfb69882197709" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 35087 } }
I20260810 07:16:26.152175 29055 raft_consensus.cc:374] T 3fdb36db7295465d93c03160c8085a31 P a34e63934b9d4bee8f54d2db3554a5eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:16:26.152261 29055 raft_consensus.cc:729] T 3fdb36db7295465d93c03160c8085a31 P a34e63934b9d4bee8f54d2db3554a5eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a34e63934b9d4bee8f54d2db3554a5eb, State: Initialized, Role: FOLLOWER
I20260810 07:16:26.152446 29055 consensus_queue.cc:260] T 3fdb36db7295465d93c03160c8085a31 P a34e63934b9d4bee8f54d2db3554a5eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a34e63934b9d4bee8f54d2db3554a5eb" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 42725 } } peers { permanent_uuid: "a68b586e8d3d42b3a2cb32b080d6291d" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 39139 } } peers { permanent_uuid: "dac9dd9a408f4699aadfb69882197709" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 35087 } }
I20260810 07:16:26.152786 29055 ts_tablet_manager.cc:1428] T 3fdb36db7295465d93c03160c8085a31 P a34e63934b9d4bee8f54d2db3554a5eb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:16:26.152921 28885 heartbeater.cc:503] Master 127.15.217.126:44607 was elected leader, sending a full tablet report...
W20260810 07:16:26.207027 28734 tablet.cc:2369] T 3fdb36db7295465d93c03160c8085a31 P a68b586e8d3d42b3a2cb32b080d6291d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:16:26.212507 29061 raft_consensus.cc:482] T 3fdb36db7295465d93c03160c8085a31 P dac9dd9a408f4699aadfb69882197709 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:16:26.212669 29061 raft_consensus.cc:504] T 3fdb36db7295465d93c03160c8085a31 P dac9dd9a408f4699aadfb69882197709 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a34e63934b9d4bee8f54d2db3554a5eb" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 42725 } } peers { permanent_uuid: "a68b586e8d3d42b3a2cb32b080d6291d" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 39139 } } peers { permanent_uuid: "dac9dd9a408f4699aadfb69882197709" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 35087 } }
I20260810 07:16:26.213116 29061 leader_election.cc:288] T 3fdb36db7295465d93c03160c8085a31 P dac9dd9a408f4699aadfb69882197709 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a34e63934b9d4bee8f54d2db3554a5eb (127.15.217.66:42725), a68b586e8d3d42b3a2cb32b080d6291d (127.15.217.65:39139)
I20260810 07:16:26.220801 29049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:26.213840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:34181 (local address 127.15.217.65:39139)
0810 07:16:26.214034 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:26.214036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:26.217062 (+ 3026us) server_negotiation.cc:407] Connection header received
0810 07:16:26.217192 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:26.217193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:26.217204 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:26.217233 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:16:26.217578 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:26.218314 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:26.219775 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:26.220355 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:26.220405 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:26.220415 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:26.220666 (+ 251us) server_negotiation.cc:299] Negotiation successful
0810 07:16:26.220727 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166}
I20260810 07:16:26.221285 29063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:26.213519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:39139 (local address 127.15.217.67:34181)
0810 07:16:26.216984 (+ 3465us) negotiation.cc:107] Waiting for socket to connect
0810 07:16:26.216997 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:16:26.217113 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:16:26.217311 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:16:26.217312 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:16:26.217315 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:16:26.217485 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:26.217488 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:26.218439 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:26.218440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:26.219665 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:26.219668 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:26.220516 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:26.220517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:26.220587 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:26.220591 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:16:26.221149 (+ 558us) client_negotiation.cc:240] Negotiation successful
0810 07:16:26.221192 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3416,"thread_start_us":98,"threads_started":1}
I20260810 07:16:26.221882 28683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fdb36db7295465d93c03160c8085a31" candidate_uuid: "dac9dd9a408f4699aadfb69882197709" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a68b586e8d3d42b3a2cb32b080d6291d" is_pre_election: true
I20260810 07:16:26.222136 28683 raft_consensus.cc:2454] T 3fdb36db7295465d93c03160c8085a31 P a68b586e8d3d42b3a2cb32b080d6291d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dac9dd9a408f4699aadfb69882197709 in term 0.
I20260810 07:16:26.222399 28907 leader_election.cc:302] T 3fdb36db7295465d93c03160c8085a31 P dac9dd9a408f4699aadfb69882197709 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a68b586e8d3d42b3a2cb32b080d6291d, dac9dd9a408f4699aadfb69882197709; no voters:
I20260810 07:16:26.222579 29061 raft_consensus.cc:2781] T 3fdb36db7295465d93c03160c8085a31 P dac9dd9a408f4699aadfb69882197709 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:16:26.222625 29061 raft_consensus.cc:482] T 3fdb36db7295465d93c03160c8085a31 P dac9dd9a408f4699aadfb69882197709 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:16:26.222647 29061 raft_consensus.cc:3037] T 3fdb36db7295465d93c03160c8085a31 P dac9dd9a408f4699aadfb69882197709 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:16:26.222851 29037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:26.213353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:42725 (local address 127.15.217.67:58945)
0810 07:16:26.213402 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0810 07:16:26.213416 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:16:26.213465 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:16:26.214075 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:16:26.214077 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:16:26.214079 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:16:26.214160 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:26.214163 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:26.215227 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:26.215228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:26.216382 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:26.216387 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:26.222515 (+ 6128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:26.222516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:26.222622 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:26.222626 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:16:26.222687 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 07:16:26.222740 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260810 07:16:26.223116 29052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:26.213759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:58945 (local address 127.15.217.66:42725)
0810 07:16:26.213822 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:26.213825 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:26.213836 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:16:26.213872 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:26.213873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:26.213886 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:26.213962 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:16:26.214248 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:26.215086 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:26.221318 (+ 6232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:26.222101 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:26.222170 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:26.222181 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:26.222971 (+ 790us) server_negotiation.cc:299] Negotiation successful
0810 07:16:26.223031 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 07:16:26.223577 29061 raft_consensus.cc:504] T 3fdb36db7295465d93c03160c8085a31 P dac9dd9a408f4699aadfb69882197709 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a34e63934b9d4bee8f54d2db3554a5eb" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 42725 } } peers { permanent_uuid: "a68b586e8d3d42b3a2cb32b080d6291d" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 39139 } } peers { permanent_uuid: "dac9dd9a408f4699aadfb69882197709" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 35087 } }
I20260810 07:16:26.223709 29061 leader_election.cc:288] T 3fdb36db7295465d93c03160c8085a31 P dac9dd9a408f4699aadfb69882197709 [CANDIDATE]: Term 1 election: Requested vote from peers a34e63934b9d4bee8f54d2db3554a5eb (127.15.217.66:42725), a68b586e8d3d42b3a2cb32b080d6291d (127.15.217.65:39139)
I20260810 07:16:26.224097 28833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fdb36db7295465d93c03160c8085a31" candidate_uuid: "dac9dd9a408f4699aadfb69882197709" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a34e63934b9d4bee8f54d2db3554a5eb" is_pre_election: true
I20260810 07:16:26.224252 28833 raft_consensus.cc:2454] T 3fdb36db7295465d93c03160c8085a31 P a34e63934b9d4bee8f54d2db3554a5eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dac9dd9a408f4699aadfb69882197709 in term 0.
I20260810 07:16:26.224359 28832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fdb36db7295465d93c03160c8085a31" candidate_uuid: "dac9dd9a408f4699aadfb69882197709" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a34e63934b9d4bee8f54d2db3554a5eb"
I20260810 07:16:26.224408 28832 raft_consensus.cc:3037] T 3fdb36db7295465d93c03160c8085a31 P a34e63934b9d4bee8f54d2db3554a5eb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:16:26.225298 28832 raft_consensus.cc:2454] T 3fdb36db7295465d93c03160c8085a31 P a34e63934b9d4bee8f54d2db3554a5eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dac9dd9a408f4699aadfb69882197709 in term 1.
I20260810 07:16:26.226087 28910 leader_election.cc:302] T 3fdb36db7295465d93c03160c8085a31 P dac9dd9a408f4699aadfb69882197709 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34e63934b9d4bee8f54d2db3554a5eb, dac9dd9a408f4699aadfb69882197709; no voters:
I20260810 07:16:26.226344 28683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fdb36db7295465d93c03160c8085a31" candidate_uuid: "dac9dd9a408f4699aadfb69882197709" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a68b586e8d3d42b3a2cb32b080d6291d"
I20260810 07:16:26.226437 28683 raft_consensus.cc:3037] T 3fdb36db7295465d93c03160c8085a31 P a68b586e8d3d42b3a2cb32b080d6291d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:16:26.227283 28683 raft_consensus.cc:2454] T 3fdb36db7295465d93c03160c8085a31 P a68b586e8d3d42b3a2cb32b080d6291d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dac9dd9a408f4699aadfb69882197709 in term 1.
I20260810 07:16:26.227566 29061 raft_consensus.cc:2781] T 3fdb36db7295465d93c03160c8085a31 P dac9dd9a408f4699aadfb69882197709 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:16:26.227743 29061 raft_consensus.cc:686] T 3fdb36db7295465d93c03160c8085a31 P dac9dd9a408f4699aadfb69882197709 [term 1 LEADER]: Becoming Leader. State: Replica: dac9dd9a408f4699aadfb69882197709, State: Running, Role: LEADER
I20260810 07:16:26.227833 29061 consensus_queue.cc:237] T 3fdb36db7295465d93c03160c8085a31 P dac9dd9a408f4699aadfb69882197709 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a34e63934b9d4bee8f54d2db3554a5eb" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 42725 } } peers { permanent_uuid: "a68b586e8d3d42b3a2cb32b080d6291d" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 39139 } } peers { permanent_uuid: "dac9dd9a408f4699aadfb69882197709" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 35087 } }
I20260810 07:16:26.228739 28496 catalog_manager.cc:5495] T 3fdb36db7295465d93c03160c8085a31 P dac9dd9a408f4699aadfb69882197709 reported cstate change: term changed from 0 to 1, leader changed from <none> to dac9dd9a408f4699aadfb69882197709 (127.15.217.67). New cstate: current_term: 1 leader_uuid: "dac9dd9a408f4699aadfb69882197709" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a34e63934b9d4bee8f54d2db3554a5eb" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 42725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a68b586e8d3d42b3a2cb32b080d6291d" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 39139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dac9dd9a408f4699aadfb69882197709" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 35087 } health_report { overall_health: HEALTHY } } }
W20260810 07:16:26.233321 28886 tablet.cc:2369] T 3fdb36db7295465d93c03160c8085a31 P a34e63934b9d4bee8f54d2db3554a5eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:16:26.274806 29050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:26.270784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40320 (local address 127.15.217.67:35087)
0810 07:16:26.270910 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:26.270912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:26.271142 (+ 230us) server_negotiation.cc:407] Connection header received
0810 07:16:26.271197 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:26.271198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:26.271224 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:26.271295 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:16:26.271536 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:26.272322 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:26.273525 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:26.273849 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:26.273921 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:26.274160 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:16:26.274318 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:16:26.274405 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:26.274696 (+ 291us) server_negotiation.cc:299] Negotiation successful
0810 07:16:26.274719 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260810 07:16:26.287736 28683 raft_consensus.cc:1261] T 3fdb36db7295465d93c03160c8085a31 P a68b586e8d3d42b3a2cb32b080d6291d [term 1 FOLLOWER]: Refusing update from remote peer dac9dd9a408f4699aadfb69882197709: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:16:26.287837 28832 raft_consensus.cc:1261] T 3fdb36db7295465d93c03160c8085a31 P a34e63934b9d4bee8f54d2db3554a5eb [term 1 FOLLOWER]: Refusing update from remote peer dac9dd9a408f4699aadfb69882197709: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:16:26.288268 29061 consensus_queue.cc:1035] T 3fdb36db7295465d93c03160c8085a31 P dac9dd9a408f4699aadfb69882197709 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34e63934b9d4bee8f54d2db3554a5eb" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 42725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:16:26.288511 29061 consensus_queue.cc:1035] T 3fdb36db7295465d93c03160c8085a31 P dac9dd9a408f4699aadfb69882197709 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a68b586e8d3d42b3a2cb32b080d6291d" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 39139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:16:26.291443 29067 mvcc.cc:205] Tried to move back new op lower bound from 7316873979030884352 to 7316873978789838848. Current Snapshot: MvccSnapshot[applied={T|T < 7316873979030884352 or (T in {7316873979030884352})}]
I20260810 07:16:26.291966 29069 mvcc.cc:205] Tried to move back new op lower bound from 7316873979030884352 to 7316873978789838848. Current Snapshot: MvccSnapshot[applied={T|T < 7316873979030884352}]
I20260810 07:16:26.293525 28496 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:16:26.293628 28496 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:16:26.293891 29068 mvcc.cc:205] Tried to move back new op lower bound from 7316873979030884352 to 7316873978789838848. Current Snapshot: MvccSnapshot[applied={T|T < 7316873979030884352 or (T in {7316873979030884352})}]
I20260810 07:16:26.294451 28496 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d556fcc65dce4e5785e1a46a93db8fcd: Sending DeleteTablet for 3 replicas of tablet 3fdb36db7295465d93c03160c8085a31
I20260810 07:16:26.294821 28657 tablet_service.cc:1514] Processing DeleteTablet for tablet 3fdb36db7295465d93c03160c8085a31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:16:26 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:41236
I20260810 07:16:26.295159 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 28586
I20260810 07:16:26.298339 28793 tablet_service.cc:1514] Processing DeleteTablet for tablet 3fdb36db7295465d93c03160c8085a31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:16:26 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:56898
I20260810 07:16:26.298568 28957 tablet_service.cc:1514] Processing DeleteTablet for tablet 3fdb36db7295465d93c03160c8085a31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:16:26 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:40306
I20260810 07:16:26.298358 28811 tablet_service.cc:1514] Processing DeleteTablet for tablet 3fdb36db7295465d93c03160c8085a31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:16:26 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:56898
I20260810 07:16:26.299217 29078 tablet_replica.cc:330] T 3fdb36db7295465d93c03160c8085a31 P dac9dd9a408f4699aadfb69882197709: stopping tablet replica
I20260810 07:16:26.299340 29078 raft_consensus.cc:2229] T 3fdb36db7295465d93c03160c8085a31 P dac9dd9a408f4699aadfb69882197709 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:16:26.299448 29078 raft_consensus.cc:2258] T 3fdb36db7295465d93c03160c8085a31 P dac9dd9a408f4699aadfb69882197709 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:16:26.299619 29077 tablet_replica.cc:330] T 3fdb36db7295465d93c03160c8085a31 P a34e63934b9d4bee8f54d2db3554a5eb: stopping tablet replica
I20260810 07:16:26.299724 29077 raft_consensus.cc:2229] T 3fdb36db7295465d93c03160c8085a31 P a34e63934b9d4bee8f54d2db3554a5eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:16:26.299832 29077 raft_consensus.cc:2258] T 3fdb36db7295465d93c03160c8085a31 P a34e63934b9d4bee8f54d2db3554a5eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:16:26.300279 29078 ts_tablet_manager.cc:1905] T 3fdb36db7295465d93c03160c8085a31 P dac9dd9a408f4699aadfb69882197709: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:16:26.301864 29077 ts_tablet_manager.cc:1905] T 3fdb36db7295465d93c03160c8085a31 P a34e63934b9d4bee8f54d2db3554a5eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:16:26.303144 29078 ts_tablet_manager.cc:1918] T 3fdb36db7295465d93c03160c8085a31 P dac9dd9a408f4699aadfb69882197709: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:16:26.303227 29078 log.cc:1196] T 3fdb36db7295465d93c03160c8085a31 P dac9dd9a408f4699aadfb69882197709: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-2/wal/wals/3fdb36db7295465d93c03160c8085a31
I20260810 07:16:26.303171 29077 ts_tablet_manager.cc:1918] T 3fdb36db7295465d93c03160c8085a31 P a34e63934b9d4bee8f54d2db3554a5eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:16:26.303411 29077 log.cc:1196] T 3fdb36db7295465d93c03160c8085a31 P a34e63934b9d4bee8f54d2db3554a5eb: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346081759552-16229-0/minicluster-data/ts-1/wal/wals/3fdb36db7295465d93c03160c8085a31
I20260810 07:16:26.303551 29078 ts_tablet_manager.cc:1939] T 3fdb36db7295465d93c03160c8085a31 P dac9dd9a408f4699aadfb69882197709: Deleting consensus metadata
I20260810 07:16:26.303828 29077 ts_tablet_manager.cc:1939] T 3fdb36db7295465d93c03160c8085a31 P a34e63934b9d4bee8f54d2db3554a5eb: Deleting consensus metadata
I20260810 07:16:26.304039 28482 catalog_manager.cc:4841] TS dac9dd9a408f4699aadfb69882197709 (127.15.217.67:35087): tablet 3fdb36db7295465d93c03160c8085a31 (table test-table [id=4f93d0e446ab418daedbe5584682e411]) successfully deleted
I20260810 07:16:26.304315 28484 catalog_manager.cc:4841] TS a34e63934b9d4bee8f54d2db3554a5eb (127.15.217.66:42725): tablet 3fdb36db7295465d93c03160c8085a31 (table test-table [id=4f93d0e446ab418daedbe5584682e411]) successfully deleted
W20260810 07:16:26.304538 28484 catalog_manager.cc:4808] TS a34e63934b9d4bee8f54d2db3554a5eb (127.15.217.66:42725): delete failed for tablet 3fdb36db7295465d93c03160c8085a31 because the tablet was not found. No further retry: Not found: Tablet not found: 3fdb36db7295465d93c03160c8085a31
W20260810 07:16:26.310053 28481 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.217.65:39139 (error 108)
I20260810 07:16:26.310310 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 28739
I20260810 07:16:26.325453 29047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:26.325294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:39139 (local address 127.0.0.1:41250)
0810 07:16:26.325353 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0810 07:16:26.325395 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.65:39139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260810 07:16:26.325805 28481 catalog_manager.cc:4584] TS a68b586e8d3d42b3a2cb32b080d6291d (127.15.217.65:39139): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3fdb36db7295465d93c03160c8085a31: Network error: Client connection negotiation failed: client connection to 127.15.217.65:39139: connect: Connection refused (error 111)
I20260810 07:16:26.330958 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 28890
I20260810 07:16:26.339040 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 28461
I20260810 07:16:26.349781 16229 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 07:16:26.694800 16229 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 07:16:26.695000 16229 mini_ranger.cc:66] Stopping Ranger...
I20260810 07:16:27.052814 16229 mini_ranger.cc:68] Stopped Ranger
2026-08-10 07:16:27.053 UTC [27992] LOG: received smart shutdown request
2026-08-10 07:16:27.057 UTC [27992] LOG: background worker "logical replication launcher" (PID 28001) exited with exit code 1
2026-08-10 07:16:27.058 UTC [27996] LOG: shutting down
2026-08-10 07:16:27.242 UTC [27992] LOG: database system is shut down
2026-08-10T07:16:27Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47539 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:16:27 dist-test-slave-10pc krb5kdc[29089](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:16:27 dist-test-slave-10pc krb5kdc[29089](info): set up 2 sockets
Aug 10 07:16:27 dist-test-slave-10pc krb5kdc[29089](info): commencing operation
krb5kdc: starting...
W20260810 07:16:28.599109 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.004s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:16:28 dist-test-slave-10pc krb5kdc[29089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346188, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:16:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:16:28Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.15.217.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.15.217.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.15.217.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/rangeradmin_127.15.217.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.15.217.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/rangeradmin_127.15.217.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.15.217.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.15.217.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.15.217.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/rangerlookup_127.15.217.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.15.217.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/rangerlookup_127.15.217.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.217.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.15.217.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.217.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/HTTP_127.15.217.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.217.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/HTTP_127.15.217.84@KRBTEST.COM.keytab.
I20260810 07:16:28.781282 16229 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskvoWU8l/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/postgres -l logfile start
2026-08-10 07:16:31.300 UTC [29123] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 07:16:31.300 UTC [29123] LOG: listening on IPv4 address "127.15.217.84", port 57445
2026-08-10 07:16:31.304 UTC [29123] LOG: listening on Unix socket "/tmp/.s.PGSQL.57445"
2026-08-10 07:16:31.326 UTC [29127] LOG: database system was shut down at 2026-08-10 07:16:29 UTC
2026-08-10 07:16:31.330 UTC [29123] LOG: database system is ready to accept connections
I20260810 07:16:32.327642 16229 mini_postgres.cc:96] Postgres bound to 57445
2026-08-10 07:16:32.335 UTC [29135] FATAL: database "slave" does not exist
127.15.217.84:57445 - accepting connections
I20260810 07:16:32.335932 16229 mini_ranger.cc:161] Starting Ranger...
I20260810 07:16:32.352722 16229 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 07:16:32.834882 16229 mini_ranger.cc:87] Created ranger Postgres database
I20260810 07:16:32.834980 16229 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ranger-admin
2026-08-10 07:16:33,127 [I] DB FLAVOR :POSTGRES
2026-08-10 07:16:33,127 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 07:16:33,127 [I] Checking connection..
2026-08-10 07:16:33,128 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:57445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 07:16:33,379 [I] Checking connection passed.
2026-08-10 07:16:33,379 [I] --------- Verifying version history table ---------
2026-08-10 07:16:33,380 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:57445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 07:16:33,654 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 07:16:33,654 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:57445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 07:16:33,911 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 07:16:33,911 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 07:16:33,911 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:57445/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 07:16:34.175 UTC [29206] WARNING: there is no transaction in progress
2026-08-10 07:16:34,182 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 07:16:34,182 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:57445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 07:16:34,423 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 07:16:34,424 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 07:16:34,424 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:57445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 07:16:34,703 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:57445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 07:16:34,976 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:57445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-10pc.c.gcp-upstream.internal','N') ;"
2026-08-10 07:16:35,207 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:57445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 07:16:35,456 [I] Table x_portal_user does not exist in database ranger
2026-08-10 07:16:35,456 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:57445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 07:16:35,725 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 07:16:35,725 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:57445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 07:16:35,963 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:57445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 07:16:36,226 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 07:16:36,227 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:57445/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 07:16:37.246 UTC [29326] WARNING: there is no transaction in progress
2026-08-10 07:16:37.262 UTC [29326] WARNING: there is no transaction in progress
2026-08-10 07:16:37.278 UTC [29326] WARNING: there is no transaction in progress
2026-08-10 07:16:37.294 UTC [29326] WARNING: there is no transaction in progress
2026-08-10 07:16:37.309 UTC [29326] WARNING: there is no transaction in progress
2026-08-10 07:16:37.327 UTC [29326] WARNING: there is no transaction in progress
2026-08-10 07:16:37.396 UTC [29326] WARNING: there is no transaction in progress
2026-08-10 07:16:37.403 UTC [29326] WARNING: there is no transaction in progress
2026-08-10 07:16:37.417 UTC [29326] WARNING: there is no transaction in progress
2026-08-10 07:16:37.427 UTC [29326] WARNING: there is no transaction in progress
2026-08-10 07:16:37.438 UTC [29326] WARNING: there is no transaction in progress
2026-08-10 07:16:37.452 UTC [29326] WARNING: there is no transaction in progress
2026-08-10 07:16:37.462 UTC [29326] WARNING: there is no transaction in progress
2026-08-10 07:16:37.469 UTC [29326] WARNING: there is no transaction in progress
2026-08-10 07:16:37.476 UTC [29326] WARNING: there is no transaction in progress
2026-08-10 07:16:37.483 UTC [29326] WARNING: there is no transaction in progress
2026-08-10 07:16:38,172 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 07:16:38,173 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:57445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 07:16:38,415 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 07:16:38,416 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:57445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 07:16:38,670 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 07:16:38,671 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:57445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 07:16:38,925 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 07:16:38,926 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:57445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 07:16:39,187 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:57445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 07:16:39,431 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:57445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-10pc.c.gcp-upstream.internal';"
2026-08-10 07:16:39,694 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 07:16:39,694 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:57445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 07:16:39,950 [I] DB_PATCHES have already been applied
I20260810 07:16:39.956116 16229 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ranger-admin:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskvoWU8l/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 07:16:39.956211 16229 mini_ranger.cc:193] Using host: 127.15.217.84
I20260810 07:16:39.960366 16229 mini_ranger.cc:227] Ranger admin URL: http://127.15.217.84:49887
Aug 10, 2026 7:16:40 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 7:16:40 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 7:16:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 7:16:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 7:16:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 7:16:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 7:16:40 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 07:16:41.142647 16229 mini_ranger.cc:160] Time spent starting Ranger: real 8.807s user 0.002s sys 0.010s
Aug 10, 2026 7:17:01 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 7:17:01 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 10, 2026 7:17:01 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 7:17:02 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 7:17:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 7:17:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 7:17:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 7:17:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 7:17:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 7:17:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 7:17:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 7:17:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 7:17:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 7:17:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 7:17:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 7:17:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 7:17:02 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 7:17:02 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 07:17:03.562465 16229 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.15.217.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.15.217.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.15.217.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/rangerkms_127.15.217.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.15.217.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/rangerkms_127.15.217.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.217.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.15.217.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.217.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/HTTP_127.15.217.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.217.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/HTTP_127.15.217.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 07:17:03 dist-test-slave-10pc krb5kdc[29089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346223, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 07:17:03.658859 16229 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 07:17:03.670090 16229 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 07:17:04.562737 16229 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 07:17:04.562812 16229 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ranger-kms
I20260810 07:17:04.562825 16229 mini_ranger_kms.cc:227] Using postgres at 127.15.217.84:57445
2026-08-10 07:17:04,741 [I] DB FLAVOR :POSTGRES
2026-08-10 07:17:04,742 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 07:17:04,742 [I] Checking connection
2026-08-10 07:17:04,742 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:57445/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 07:17:05,016 [I] connection success
2026-08-10 07:17:05,016 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 07:17:05,016 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 07:17:05,017 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:57445/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 07:17:05,272 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 07:17:05,272 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 07:17:05,273 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 07:17:05,273 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.217.84:57445/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 07:17:05,559 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 07:17:06.060813 16229 mini_ranger_kms.cc:310] Created kms service
I20260810 07:17:06.255888 16229 mini_ranger_kms.cc:326] Created kudu user
I20260810 07:17:06.302404 16229 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 07:17:06 dist-test-slave-10pc krb5kdc[29089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346223, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.15.217.84@KRBTEST.COM
I20260810 07:17:06.731051 16229 mini_ranger_kms.cc:384] Added ranger policy
I20260810 07:17:06.731305 16229 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ranger-kms:/tmp/dist-test-taskvoWU8l/build/release/bin/postgresql.jar:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskvoWU8l/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 07:17:06.731333 16229 mini_ranger_kms.cc:240] Using host: 127.15.217.84
I20260810 07:17:06.736251 16229 mini_ranger_kms.cc:276] Ranger KMS PID: 29523
I20260810 07:17:06.736352 16229 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.15.217.84:40247
Aug 10, 2026 7:17:07 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 7:17:07 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 7:17:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 7:17:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 7:17:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 7:17:08 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 07:17:09 dist-test-slave-10pc krb5kdc[29089](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786346229, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.217.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:09.211193 16229 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.552s user 0.015s sys 0.148s
I20260810 07:17:09.211310 16229 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 07:17:09.211360 16229 mini_ranger_kms.cc:401] 127.15.217.84:40247/kms/v1/keys
Aug 10 07:17:09 dist-test-slave-10pc krb5kdc[29089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346229, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.217.84@KRBTEST.COM for HTTP/127.15.217.84@KRBTEST.COM
Aug 10 07:17:09 dist-test-slave-10pc krb5kdc[29089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346229, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.217.84@KRBTEST.COM for HTTP/127.15.217.84@KRBTEST.COM
Aug 10 07:17:12 dist-test-slave-10pc krb5kdc[29089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.15.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
I20260810 07:17:12.213646 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:38731
--webserver_interface=127.15.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:38125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:38731
--ranger_config_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.217.84:40247/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:17:12.318269 29576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:12.318539 29576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:12.318578 29576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:12.319766 29576 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:17:12.319837 29576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:12.319861 29576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:12.319882 29576 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:17:12.319912 29576 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:17:12.321581 29576 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:38125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/master-0/wal
--ranger_kms_url=127.15.217.84:40247/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:38731
--ranger_config_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:38731
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:17:12.321873 29576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:12.322144 29576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:12.327643 29581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:12.327801 29576 server_base.cc:1034] running on GCE node
W20260810 07:17:12.327659 29582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:12.327659 29586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:12.328284 29576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:12.328685 29576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:12.329870 29576 hybrid_clock.cc:648] HybridClock initialized: now 1786346232329854 us; error 29 us; skew 500 ppm
Aug 10 07:17:12 dist-test-slave-10pc krb5kdc[29089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346232, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:12.337738 29576 init.cc:377] Logged in from keytab as kudu/127.15.217.126@KRBTEST.COM (short username kudu)
I20260810 07:17:12.338282 29576 webserver.cc:460] Webserver started at http://127.15.217.126:41161/ using document root <none> and password file <none>
I20260810 07:17:12.338529 29576 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:12.338599 29576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:12.338763 29576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 07:17:12 dist-test-slave-10pc krb5kdc[29089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346232, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for HTTP/127.15.217.84@KRBTEST.COM
I20260810 07:17:12.402577 29576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "db37adc733684cc1aa5a0dcf1794eca0"
format_stamp: "Formatted at 2026-08-10 07:17:12 on dist-test-slave-10pc"
server_key: "fef988f0c74e64c1e828364943af5fe0"
server_key_iv: "67bbb854913f2a681f10ef6459d1ebd3"
server_key_version: "kuduclusterkey@0"
I20260810 07:17:12.403047 29576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "db37adc733684cc1aa5a0dcf1794eca0"
format_stamp: "Formatted at 2026-08-10 07:17:12 on dist-test-slave-10pc"
server_key: "fef988f0c74e64c1e828364943af5fe0"
server_key_iv: "67bbb854913f2a681f10ef6459d1ebd3"
server_key_version: "kuduclusterkey@0"
I20260810 07:17:12.404593 29576 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:17:12.420636 29594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:12.420907 29576 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:17:12.421022 29576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "db37adc733684cc1aa5a0dcf1794eca0"
format_stamp: "Formatted at 2026-08-10 07:17:12 on dist-test-slave-10pc"
server_key: "fef988f0c74e64c1e828364943af5fe0"
server_key_iv: "67bbb854913f2a681f10ef6459d1ebd3"
server_key_version: "kuduclusterkey@0"
I20260810 07:17:12.421150 29576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:12.441900 29576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:12.444113 29576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:12.444285 29576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:12.449491 29576 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:38731
I20260810 07:17:12.449518 29650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:38731 every 8 connection(s)
I20260810 07:17:12.449942 29576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:17:12.450860 29651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:17:12.452180 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 29576
I20260810 07:17:12.452307 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:17:12.452526 16229 external_mini_cluster.cc:1428] Setting key d4d3a2daed644eebc2021c63698575ca
I20260810 07:17:12.454795 29651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0: Bootstrap starting.
I20260810 07:17:12.455632 29651 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0: Neither blocks nor log segments found. Creating new log.
I20260810 07:17:12.455957 29651 log.cc:824] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:12.456866 29651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0: No bootstrap required, opened a new log
Aug 10 07:17:12 dist-test-slave-10pc krb5kdc[29089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:17:12.459450 29651 raft_consensus.cc:348] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db37adc733684cc1aa5a0dcf1794eca0" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 38731 } }
I20260810 07:17:12.459661 29651 raft_consensus.cc:374] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:17:12.459746 29651 raft_consensus.cc:729] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db37adc733684cc1aa5a0dcf1794eca0, State: Initialized, Role: FOLLOWER
I20260810 07:17:12.459900 29651 consensus_queue.cc:260] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db37adc733684cc1aa5a0dcf1794eca0" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 38731 } }
I20260810 07:17:12.460000 29651 raft_consensus.cc:388] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:17:12.460083 29651 raft_consensus.cc:482] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:17:12.460140 29651 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:17:12.460928 29651 raft_consensus.cc:504] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db37adc733684cc1aa5a0dcf1794eca0" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 38731 } }
I20260810 07:17:12.461054 29651 leader_election.cc:302] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db37adc733684cc1aa5a0dcf1794eca0; no voters:
I20260810 07:17:12.461241 29651 leader_election.cc:288] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:17:12.461290 29656 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:17:12.461536 29656 raft_consensus.cc:686] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0 [term 1 LEADER]: Becoming Leader. State: Replica: db37adc733684cc1aa5a0dcf1794eca0, State: Running, Role: LEADER
I20260810 07:17:12.461622 29651 sys_catalog.cc:564] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:17:12.461696 29656 consensus_queue.cc:237] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db37adc733684cc1aa5a0dcf1794eca0" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 38731 } }
I20260810 07:17:12.462605 29658 sys_catalog.cc:455] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db37adc733684cc1aa5a0dcf1794eca0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db37adc733684cc1aa5a0dcf1794eca0" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 38731 } } }
I20260810 07:17:12.462764 29658 sys_catalog.cc:458] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0 [sys.catalog]: This master's current role is: LEADER
I20260810 07:17:12.463495 29666 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:17:12.464030 29657 sys_catalog.cc:455] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db37adc733684cc1aa5a0dcf1794eca0. Latest consensus state: current_term: 1 leader_uuid: "db37adc733684cc1aa5a0dcf1794eca0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db37adc733684cc1aa5a0dcf1794eca0" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 38731 } } }
I20260810 07:17:12.464267 29657 sys_catalog.cc:458] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0 [sys.catalog]: This master's current role is: LEADER
I20260810 07:17:12.463658 29651 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 07:17:12.464319 29654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:12.453115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50148 (local address 127.15.217.126:38731)
0810 07:17:12.453294 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:12.453304 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:12.453340 (+ 36us) server_negotiation.cc:407] Connection header received
0810 07:17:12.453818 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:12.453826 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:12.453875 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:12.454030 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:12.454721 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:12.455809 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:12.456849 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:12.457329 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:12.457457 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:12.459964 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:12.459969 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:12.459969 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:12.459982 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:12.462104 (+ 2122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:12.462551 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:12.462554 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:12.462555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:12.462672 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:12.462920 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:12.462921 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:12.462921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:12.463248 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:12.463407 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:12.463984 (+ 577us) server_negotiation.cc:299] Negotiation successful
0810 07:17:12.464116 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":72,"threads_started":1}
I20260810 07:17:12.464267 29666 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:17:12.469959 29666 catalog_manager.cc:1347] Generated new cluster ID: 7432e15f7984491f9e0c93f66227f410
I20260810 07:17:12.470118 29666 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:17:12.485948 29666 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:17:12.486562 29666 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:17:12.497368 29666 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0: Generated new TSK 0
I20260810 07:17:12.497598 29666 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab.
Entry for principal kudu/127.15.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab.
I20260810 07:17:13.200736 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.217.84:40247/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.65:0
--local_ip_for_outbound_sockets=127.15.217.65
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:38731
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:38125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:17:13.322479 29699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:13.322783 29699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:13.322820 29699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:13.324587 29699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:13.324656 29699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:13.324694 29699 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.65
I20260810 07:17:13.326501 29699 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:38125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.15.217.84:40247/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:38731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.65
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:17:13.326788 29699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:13.327064 29699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:13.332676 29705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:13.332676 29706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:13.333066 29708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:13.333206 29699 server_base.cc:1034] running on GCE node
I20260810 07:17:13.333439 29699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:13.333745 29699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:13.334932 29699 hybrid_clock.cc:648] HybridClock initialized: now 1786346233334924 us; error 32 us; skew 500 ppm
Aug 10 07:17:13 dist-test-slave-10pc krb5kdc[29089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346233, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:13.337990 29699 init.cc:377] Logged in from keytab as kudu/127.15.217.65@KRBTEST.COM (short username kudu)
I20260810 07:17:13.338578 29699 webserver.cc:460] Webserver started at http://127.15.217.65:33901/ using document root <none> and password file <none>
I20260810 07:17:13.338883 29699 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:13.338948 29699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:13.339089 29699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 07:17:13 dist-test-slave-10pc krb5kdc[29089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346233, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for HTTP/127.15.217.84@KRBTEST.COM
I20260810 07:17:13.352074 29699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-0/data/instance:
uuid: "40c1e67b940a4439ba04220a0d7f1e90"
format_stamp: "Formatted at 2026-08-10 07:17:13 on dist-test-slave-10pc"
server_key: "f251ab584e5dc57d58391e53da67b537"
server_key_iv: "d7e1a1be7b913b8838220df5d2e3318e"
server_key_version: "kuduclusterkey@0"
I20260810 07:17:13.352669 29699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance:
uuid: "40c1e67b940a4439ba04220a0d7f1e90"
format_stamp: "Formatted at 2026-08-10 07:17:13 on dist-test-slave-10pc"
server_key: "f251ab584e5dc57d58391e53da67b537"
server_key_iv: "d7e1a1be7b913b8838220df5d2e3318e"
server_key_version: "kuduclusterkey@0"
I20260810 07:17:13.354667 29699 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:17:13.375214 29719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:13.375492 29699 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260810 07:17:13.375595 29699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-0/wal
uuid: "40c1e67b940a4439ba04220a0d7f1e90"
format_stamp: "Formatted at 2026-08-10 07:17:13 on dist-test-slave-10pc"
server_key: "f251ab584e5dc57d58391e53da67b537"
server_key_iv: "d7e1a1be7b913b8838220df5d2e3318e"
server_key_version: "kuduclusterkey@0"
I20260810 07:17:13.375700 29699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:13.400782 29699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:13.403492 29699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:13.403680 29699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:13.403985 29699 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:17:13.404444 29699 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:17:13.404524 29699 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:13.404615 29699 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:17:13.404649 29699 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:13.411765 29699 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.65:43377
I20260810 07:17:13.412356 29699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
I20260810 07:17:13.413192 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 29699
I20260810 07:17:13.413295 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance
I20260810 07:17:13.413520 16229 external_mini_cluster.cc:1428] Setting key d87b81726477ef5772133479f04d9f1d
I20260810 07:17:13.415740 29841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.65:43377 every 8 connection(s)
Aug 10 07:17:13 dist-test-slave-10pc krb5kdc[29089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346233, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.217.66@KRBTEST.COM; defaulting to no policy
I20260810 07:17:13.435846 29847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:13.416386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:50727 (local address 127.15.217.126:38731)
0810 07:17:13.418572 (+ 2186us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:13.418575 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:13.418587 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:17:13.418617 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:13.418618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:13.418632 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:13.419536 (+ 904us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:13.419586 (+ 50us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:13.420418 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:13.422037 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:13.422343 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:13.425329 (+ 2986us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:13.427656 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:13.427658 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:13.427658 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:13.427666 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:13.433897 (+ 6231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:13.434491 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:13.434492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:13.434492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:13.434568 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:13.434954 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:13.434956 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:13.434956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:13.435125 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:13.435194 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:13.435620 (+ 426us) server_negotiation.cc:299] Negotiation successful
0810 07:17:13.435728 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2149,"thread_start_us":109,"threads_started":1}
I20260810 07:17:13.436048 29844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:13.415224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:38731 (local address 127.15.217.65:50727)
0810 07:17:13.415888 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:13.415935 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:13.416674 (+ 739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:13.418837 (+ 2163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:13.418842 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:13.419116 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:13.419432 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:13.419439 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:13.420587 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:13.420588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:13.421906 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:13.421909 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:13.424802 (+ 2893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:13.424803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:13.425048 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:13.425710 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:13.425721 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:13.427538 (+ 1817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:13.434018 (+ 6480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:13.434027 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:13.434031 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:13.434370 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:13.434674 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:13.434676 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:13.434676 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:13.434857 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:13.435229 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:13.435239 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:13.435435 (+ 196us) client_negotiation.cc:769] Sending connection context
0810 07:17:13.435572 (+ 137us) client_negotiation.cc:240] Negotiation successful
0810 07:17:13.435767 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":93,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.66@KRBTEST.COM" created.
I20260810 07:17:13.438812 29842 heartbeater.cc:347] Connected to a master server at 127.15.217.126:38731
I20260810 07:17:13.438931 29842 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:13.439168 29842 heartbeater.cc:511] Master 127.15.217.126:38731 requested a full tablet report, sending...
I20260810 07:17:13.440784 29613 ts_manager.cc:194] Registered new tserver with Master: 40c1e67b940a4439ba04220a0d7f1e90 (127.15.217.65:43377)
I20260810 07:17:13.441908 29613 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.65@KRBTEST.COM'} at 127.15.217.65:50727
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab.
Entry for principal kudu/127.15.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab.
I20260810 07:17:13.467557 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.217.84:40247/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.66:0
--local_ip_for_outbound_sockets=127.15.217.66
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:38731
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:38125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 07:17:13 dist-test-slave-10pc krb5kdc[29089](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786346233, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 07:17:13.583747 29849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:13.584103 29849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:13.584206 29849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:13.586118 29849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:13.586182 29849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:13.586222 29849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.66
I20260810 07:17:13.588495 29849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:38125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.15.217.84:40247/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:38731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.66
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:17:13.588992 29849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:13.589339 29849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:13.596125 29859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:13.596586 29856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:13.596117 29857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:13.597178 29849 server_base.cc:1034] running on GCE node
I20260810 07:17:13.597437 29849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:13.597784 29849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:13.600544 29849 hybrid_clock.cc:648] HybridClock initialized: now 1786346233600517 us; error 42 us; skew 500 ppm
Aug 10 07:17:13 dist-test-slave-10pc krb5kdc[29089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346233, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:13.604113 29849 init.cc:377] Logged in from keytab as kudu/127.15.217.66@KRBTEST.COM (short username kudu)
I20260810 07:17:13.604771 29849 webserver.cc:460] Webserver started at http://127.15.217.66:36801/ using document root <none> and password file <none>
I20260810 07:17:13.605111 29849 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:13.605181 29849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:13.605370 29849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 07:17:13 dist-test-slave-10pc krb5kdc[29089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346233, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for HTTP/127.15.217.84@KRBTEST.COM
I20260810 07:17:13.619573 29849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-1/data/instance:
uuid: "73e7a349a5c545d986afbf96ec100697"
format_stamp: "Formatted at 2026-08-10 07:17:13 on dist-test-slave-10pc"
server_key: "6d049d9137f6c9677a194c4599cacffe"
server_key_iv: "e0d4f0f44181a6478cbf0395a03d40af"
server_key_version: "kuduclusterkey@0"
I20260810 07:17:13.620049 29849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance:
uuid: "73e7a349a5c545d986afbf96ec100697"
format_stamp: "Formatted at 2026-08-10 07:17:13 on dist-test-slave-10pc"
server_key: "6d049d9137f6c9677a194c4599cacffe"
server_key_iv: "e0d4f0f44181a6478cbf0395a03d40af"
server_key_version: "kuduclusterkey@0"
I20260810 07:17:13.622620 29849 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260810 07:17:13.641937 29867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:13.642155 29849 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.001s
I20260810 07:17:13.642272 29849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-1/wal
uuid: "73e7a349a5c545d986afbf96ec100697"
format_stamp: "Formatted at 2026-08-10 07:17:13 on dist-test-slave-10pc"
server_key: "6d049d9137f6c9677a194c4599cacffe"
server_key_iv: "e0d4f0f44181a6478cbf0395a03d40af"
server_key_version: "kuduclusterkey@0"
I20260810 07:17:13.642391 29849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:13.698405 29849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:13.701157 29849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:13.701399 29849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:13.701756 29849 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:17:13.702214 29849 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:17:13.702302 29849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:13.702366 29849 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:17:13.702417 29849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:13.711552 29849 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.66:46229
I20260810 07:17:13.712154 29849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
I20260810 07:17:13.712164 29998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.66:46229 every 8 connection(s)
I20260810 07:17:13.717026 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 29849
I20260810 07:17:13.717135 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance
I20260810 07:17:13.717336 16229 external_mini_cluster.cc:1428] Setting key 472eb7bb1ddce34d5033666fb3e0e5d4
Aug 10 07:17:13 dist-test-slave-10pc krb5kdc[29089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346233, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:17:13.725099 30001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:13.713298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:38731 (local address 127.15.217.66:56501)
0810 07:17:13.713475 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:13.713528 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:13.714271 (+ 743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:13.714709 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:13.714713 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:13.714993 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:13.715299 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:13.715306 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:13.716344 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:13.716345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:13.717697 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:13.717700 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:13.718251 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:13.718252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:13.718402 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:13.719069 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:13.719081 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:13.721178 (+ 2097us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:13.723155 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:13.723164 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:13.723167 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:13.723506 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:13.723755 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:13.723756 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:13.723756 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:13.723902 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:13.724233 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:13.724241 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:13.724442 (+ 201us) client_negotiation.cc:769] Sending connection context
0810 07:17:13.724602 (+ 160us) client_negotiation.cc:240] Negotiation successful
0810 07:17:13.724789 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":83,"threads_started":1}
I20260810 07:17:13.725898 29847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:13.713335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:56501 (local address 127.15.217.126:38731)
0810 07:17:13.713385 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:13.713387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:13.713631 (+ 244us) server_negotiation.cc:407] Connection header received
0810 07:17:13.714423 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:13.714425 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:13.714443 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:13.714493 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:13.715430 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:13.716214 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:13.717798 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:13.718096 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:13.718143 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:13.721304 (+ 3161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:13.721305 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:13.721306 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:13.721316 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:13.723052 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:13.723606 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:13.723607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:13.723608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:13.723674 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:13.723978 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:13.723979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:13.723980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:13.724129 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:13.724202 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:13.725698 (+ 1496us) server_negotiation.cc:299] Negotiation successful
0810 07:17:13.725806 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 07:17:13.726287 29999 heartbeater.cc:347] Connected to a master server at 127.15.217.126:38731
I20260810 07:17:13.726388 29999 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:13.726601 29999 heartbeater.cc:511] Master 127.15.217.126:38731 requested a full tablet report, sending...
I20260810 07:17:13.727244 29613 ts_manager.cc:194] Registered new tserver with Master: 73e7a349a5c545d986afbf96ec100697 (127.15.217.66:46229)
I20260810 07:17:13.727764 29613 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.66@KRBTEST.COM'} at 127.15.217.66:56501
WARNING: no policy specified for kudu/127.15.217.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab.
Entry for principal kudu/127.15.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab.
I20260810 07:17:13.756211 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.217.84:40247/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.67:0
--local_ip_for_outbound_sockets=127.15.217.67
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:38731
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:38125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:17:13.860581 30004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:13.860898 30004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:13.860934 30004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:13.862692 30004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:13.862756 30004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:13.862793 30004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.67
I20260810 07:17:13.864959 30004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:38125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.15.217.84:40247/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:38731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.67
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:17:13.865221 30004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:13.865509 30004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:13.871155 30009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:13.871258 30010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:13.871222 30012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:13.871574 30004 server_base.cc:1034] running on GCE node
I20260810 07:17:13.871935 30004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:13.872257 30004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:13.873425 30004 hybrid_clock.cc:648] HybridClock initialized: now 1786346233873379 us; error 66 us; skew 500 ppm
Aug 10 07:17:13 dist-test-slave-10pc krb5kdc[29089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346233, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:13.876129 30004 init.cc:377] Logged in from keytab as kudu/127.15.217.67@KRBTEST.COM (short username kudu)
I20260810 07:17:13.876672 30004 webserver.cc:460] Webserver started at http://127.15.217.67:45187/ using document root <none> and password file <none>
I20260810 07:17:13.876917 30004 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:13.876989 30004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:13.877152 30004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 07:17:13 dist-test-slave-10pc krb5kdc[29089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346233, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for HTTP/127.15.217.84@KRBTEST.COM
I20260810 07:17:13.890585 30004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-2/data/instance:
uuid: "4048b33da76d4ae9829eabf5eb25a9fb"
format_stamp: "Formatted at 2026-08-10 07:17:13 on dist-test-slave-10pc"
server_key: "86252225d7d31bc7b619b5b322bcae26"
server_key_iv: "13c5be2a78545d3150d99a7edb68a585"
server_key_version: "kuduclusterkey@0"
I20260810 07:17:13.891184 30004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance:
uuid: "4048b33da76d4ae9829eabf5eb25a9fb"
format_stamp: "Formatted at 2026-08-10 07:17:13 on dist-test-slave-10pc"
server_key: "86252225d7d31bc7b619b5b322bcae26"
server_key_iv: "13c5be2a78545d3150d99a7edb68a585"
server_key_version: "kuduclusterkey@0"
I20260810 07:17:13.893839 30004 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:17:13.908756 30019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:13.909085 30004 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:17:13.909248 30004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-2/wal
uuid: "4048b33da76d4ae9829eabf5eb25a9fb"
format_stamp: "Formatted at 2026-08-10 07:17:13 on dist-test-slave-10pc"
server_key: "86252225d7d31bc7b619b5b322bcae26"
server_key_iv: "13c5be2a78545d3150d99a7edb68a585"
server_key_version: "kuduclusterkey@0"
I20260810 07:17:13.909423 30004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:13.931833 30004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:13.934729 30004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:13.934988 30004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:13.935328 30004 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:17:13.935813 30004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:17:13.935911 30004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:13.935992 30004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:17:13.936056 30004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:13.943044 30004 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.67:46103
I20260810 07:17:13.943630 30004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
I20260810 07:17:13.953531 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 30004
I20260810 07:17:13.953691 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance
I20260810 07:17:13.953881 16229 external_mini_cluster.cc:1428] Setting key ac0f080ffdf931ed9c339f990896840c
I20260810 07:17:13.960366 30145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.67:46103 every 8 connection(s)
Aug 10 07:17:13 dist-test-slave-10pc krb5kdc[29089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346233, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:17:13.972244 29847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:13.960605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:37351 (local address 127.15.217.126:38731)
0810 07:17:13.960660 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:13.960662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:13.961103 (+ 441us) server_negotiation.cc:407] Connection header received
0810 07:17:13.961852 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:13.961853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:13.961866 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:13.961897 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:13.962820 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:13.963637 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:13.965247 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:13.965532 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:13.965575 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:13.968728 (+ 3153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:13.968730 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:13.968731 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:13.968739 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:13.970521 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:13.971053 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:13.971054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:13.971054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:13.971118 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:13.971411 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:13.971412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:13.971413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:13.971557 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:13.971614 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:13.972019 (+ 405us) server_negotiation.cc:299] Negotiation successful
0810 07:17:13.972141 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 07:17:13.973086 30149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:13.960778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:38731 (local address 127.15.217.67:37351)
0810 07:17:13.960962 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:13.961010 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:13.961713 (+ 703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:13.962104 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:13.962109 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:13.962376 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:13.962696 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:13.962702 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:13.963760 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:13.963761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:13.965151 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:13.965155 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:13.965698 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:13.965699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:13.965861 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:13.966778 (+ 917us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:13.966790 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:13.968608 (+ 1818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:13.970639 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:13.970646 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:13.970649 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:13.970955 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:13.971192 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:13.971193 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:13.971193 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:13.971334 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:13.971666 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:13.971673 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:13.971848 (+ 175us) client_negotiation.cc:769] Sending connection context
0810 07:17:13.972637 (+ 789us) client_negotiation.cc:240] Negotiation successful
0810 07:17:13.972827 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":113,"threads_started":1}
I20260810 07:17:13.977692 30147 heartbeater.cc:347] Connected to a master server at 127.15.217.126:38731
I20260810 07:17:13.977807 30147 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:13.978026 30147 heartbeater.cc:511] Master 127.15.217.126:38731 requested a full tablet report, sending...
I20260810 07:17:13.978471 29613 ts_manager.cc:194] Registered new tserver with Master: 4048b33da76d4ae9829eabf5eb25a9fb (127.15.217.67:46103)
I20260810 07:17:13.978999 29613 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.67@KRBTEST.COM'} at 127.15.217.67:37351
I20260810 07:17:13.987149 16229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:17:13.987979 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 29699
I20260810 07:17:13.996965 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 29849
I20260810 07:17:14.012933 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 30004
I20260810 07:17:14.023156 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 29576
I20260810 07:17:14.035852 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:38731
--webserver_interface=127.15.217.126
--webserver_port=41161
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:38125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:38731
--ranger_config_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.217.84:40247/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.217.84:40247/kms with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:17:14.120855 30150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:14.121064 30150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:14.121085 30150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:14.122262 30150 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:17:14.122309 30150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:14.122327 30150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:14.122340 30150 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:17:14.122382 30150 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:17:14.123966 30150 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:38125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.217.84:40247/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:38731
--ranger_config_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:38731
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=41161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:17:14.124234 30150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:14.124485 30150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:14.129506 30157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:14.129529 30159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:14.129514 30156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:14.129834 30150 server_base.cc:1034] running on GCE node
I20260810 07:17:14.130088 30150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:14.130383 30150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:14.131553 30150 hybrid_clock.cc:648] HybridClock initialized: now 1786346234131536 us; error 47 us; skew 500 ppm
Aug 10 07:17:14 dist-test-slave-10pc krb5kdc[29089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346234, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:14.134100 30150 init.cc:377] Logged in from keytab as kudu/127.15.217.126@KRBTEST.COM (short username kudu)
I20260810 07:17:14.134581 30150 webserver.cc:460] Webserver started at http://127.15.217.126:41161/ using document root <none> and password file <none>
I20260810 07:17:14.134833 30150 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:14.134915 30150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 07:17:14 dist-test-slave-10pc krb5kdc[29089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346234, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for HTTP/127.15.217.84@KRBTEST.COM
I20260810 07:17:14.207453 30150 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:17:14.208312 30167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:14.208557 30150 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:17:14.208664 30150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "db37adc733684cc1aa5a0dcf1794eca0"
format_stamp: "Formatted at 2026-08-10 07:17:12 on dist-test-slave-10pc"
server_key: "fef988f0c74e64c1e828364943af5fe0"
server_key_iv: "67bbb854913f2a681f10ef6459d1ebd3"
server_key_version: "kuduclusterkey@0"
I20260810 07:17:14.209183 30150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:14.218338 30150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:14.220077 30150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:14.220212 30150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:14.224591 30150 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:38731
I20260810 07:17:14.224651 30221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:38731 every 8 connection(s)
I20260810 07:17:14.225050 30150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:17:14.226301 30222 sys_catalog.cc:263] Verifying existing consensus state
I20260810 07:17:14.226999 30222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0: Bootstrap starting.
I20260810 07:17:14.233325 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 30150
I20260810 07:17:14.233700 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.217.84:40247/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.65:43377
--local_ip_for_outbound_sockets=127.15.217.65
--tserver_master_addrs=127.15.217.126:38731
--webserver_port=33901
--webserver_interface=127.15.217.65
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:38125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.217.84:40247/kms with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:17:14.235634 30222 log.cc:824] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:14.237879 30222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:17:14.238261 30222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0: Bootstrap complete.
I20260810 07:17:14.242462 30222 raft_consensus.cc:348] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db37adc733684cc1aa5a0dcf1794eca0" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 38731 } }
I20260810 07:17:14.242846 30222 raft_consensus.cc:729] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: db37adc733684cc1aa5a0dcf1794eca0, State: Initialized, Role: FOLLOWER
I20260810 07:17:14.243100 30222 consensus_queue.cc:260] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db37adc733684cc1aa5a0dcf1794eca0" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 38731 } }
I20260810 07:17:14.243209 30222 raft_consensus.cc:388] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:17:14.243280 30222 raft_consensus.cc:482] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:17:14.243340 30222 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0 [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:17:14.245110 30222 raft_consensus.cc:504] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db37adc733684cc1aa5a0dcf1794eca0" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 38731 } }
I20260810 07:17:14.245234 30222 leader_election.cc:302] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db37adc733684cc1aa5a0dcf1794eca0; no voters:
I20260810 07:17:14.245412 30222 leader_election.cc:288] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 07:17:14.245568 30226 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 07:17:14.245710 30222 sys_catalog.cc:564] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:17:14.245857 30226 raft_consensus.cc:686] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0 [term 2 LEADER]: Becoming Leader. State: Replica: db37adc733684cc1aa5a0dcf1794eca0, State: Running, Role: LEADER
I20260810 07:17:14.246040 30226 consensus_queue.cc:237] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db37adc733684cc1aa5a0dcf1794eca0" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 38731 } }
I20260810 07:17:14.246393 30226 sys_catalog.cc:455] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db37adc733684cc1aa5a0dcf1794eca0. Latest consensus state: current_term: 2 leader_uuid: "db37adc733684cc1aa5a0dcf1794eca0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db37adc733684cc1aa5a0dcf1794eca0" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 38731 } } }
I20260810 07:17:14.246483 30226 sys_catalog.cc:458] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0 [sys.catalog]: This master's current role is: LEADER
I20260810 07:17:14.246532 30227 sys_catalog.cc:455] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "db37adc733684cc1aa5a0dcf1794eca0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db37adc733684cc1aa5a0dcf1794eca0" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 38731 } } }
I20260810 07:17:14.246613 30227 sys_catalog.cc:458] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0 [sys.catalog]: This master's current role is: LEADER
I20260810 07:17:14.246735 30235 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:17:14.247318 30235 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:17:14.247480 30235 catalog_manager.cc:1259] Loaded cluster ID: 7432e15f7984491f9e0c93f66227f410
I20260810 07:17:14.247530 30235 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:17:14.247810 30222 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 07:17:14.248831 30235 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:17:14.249042 30235 catalog_manager.cc:5879] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0: Loaded TSK: 0
I20260810 07:17:14.249280 30235 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 07:17:14.371572 30224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:14.371892 30224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:14.371937 30224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:14.373910 30224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:14.374006 30224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:14.374054 30224 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.65
I20260810 07:17:14.376719 30224 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:38125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.217.84:40247/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.65:43377
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.217.65
--webserver_port=33901
--tserver_master_addrs=127.15.217.126:38731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.65
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:17:14.377039 30224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:14.377362 30224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:14.383952 30268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:14.384179 30265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:14.383970 30266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:14.384575 30224 server_base.cc:1034] running on GCE node
I20260810 07:17:14.384793 30224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:14.385076 30224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:14.386243 30224 hybrid_clock.cc:648] HybridClock initialized: now 1786346234386214 us; error 48 us; skew 500 ppm
Aug 10 07:17:14 dist-test-slave-10pc krb5kdc[29089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346234, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:14.389230 30224 init.cc:377] Logged in from keytab as kudu/127.15.217.65@KRBTEST.COM (short username kudu)
I20260810 07:17:14.389741 30224 webserver.cc:460] Webserver started at http://127.15.217.65:33901/ using document root <none> and password file <none>
I20260810 07:17:14.390009 30224 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:14.390107 30224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 07:17:14 dist-test-slave-10pc krb5kdc[29089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346234, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for HTTP/127.15.217.84@KRBTEST.COM
I20260810 07:17:14.414644 30224 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:17:14.415818 30277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:14.416003 30224 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:17:14.416098 30224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-0/wal
uuid: "40c1e67b940a4439ba04220a0d7f1e90"
format_stamp: "Formatted at 2026-08-10 07:17:13 on dist-test-slave-10pc"
server_key: "f251ab584e5dc57d58391e53da67b537"
server_key_iv: "d7e1a1be7b913b8838220df5d2e3318e"
server_key_version: "kuduclusterkey@0"
I20260810 07:17:14.416625 30224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:14.433915 30224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:14.435942 30224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:14.436160 30224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:14.436457 30224 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:17:14.436949 30224 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:17:14.437012 30224 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:14.437057 30224 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:17:14.437083 30224 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:14.444795 30224 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.65:43377
I20260810 07:17:14.444962 30399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.65:43377 every 8 connection(s)
I20260810 07:17:14.445446 30224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
I20260810 07:17:14.453354 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 30224
I20260810 07:17:14.453691 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.217.84:40247/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.66:46229
--local_ip_for_outbound_sockets=127.15.217.66
--tserver_master_addrs=127.15.217.126:38731
--webserver_port=36801
--webserver_interface=127.15.217.66
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:38125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.217.84:40247/kms with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 07:17:14 dist-test-slave-10pc krb5kdc[29089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346234, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:17:14.462038 30402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:14.446328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:38731 (local address 127.15.217.65:39559)
0810 07:17:14.446532 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:14.446587 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:14.447264 (+ 677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:14.447819 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:14.447824 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:14.448069 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:14.448373 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:14.448380 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:14.450089 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:14.450091 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:14.450896 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:14.450898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:14.451624 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:14.451626 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:14.451763 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:14.455442 (+ 3679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:14.455456 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:14.458026 (+ 2570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:14.460150 (+ 2124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:14.460159 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:14.460162 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:14.460416 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:14.460749 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:14.460750 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:14.460750 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:14.460878 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:14.461311 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:14.461319 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:14.461476 (+ 157us) client_negotiation.cc:769] Sending connection context
0810 07:17:14.461601 (+ 125us) client_negotiation.cc:240] Negotiation successful
0810 07:17:14.461789 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":94,"threads_started":1}
I20260810 07:17:14.462026 30404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:14.446452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:39559 (local address 127.15.217.126:38731)
0810 07:17:14.446645 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:14.446654 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:14.446744 (+ 90us) server_negotiation.cc:407] Connection header received
0810 07:17:14.447557 (+ 813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:14.447564 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:14.447615 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:14.447731 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:14.448704 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:14.449966 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:14.450995 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:14.451500 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:14.451629 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:14.458189 (+ 6560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:14.458196 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:14.458196 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:14.458212 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:14.460017 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:14.460547 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:14.460551 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:14.460552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:14.460663 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:14.460948 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:14.460949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:14.460949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:14.461232 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:14.461334 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:14.461660 (+ 326us) server_negotiation.cc:299] Negotiation successful
0810 07:17:14.461790 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":91,"threads_started":1}
I20260810 07:17:14.463222 30400 heartbeater.cc:347] Connected to a master server at 127.15.217.126:38731
I20260810 07:17:14.463364 30400 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:14.463583 30400 heartbeater.cc:511] Master 127.15.217.126:38731 requested a full tablet report, sending...
W20260810 07:17:14.464557 30400 heartbeater.cc:644] Failed to heartbeat to 127.15.217.126:38731 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 07:17:14.464654 30400 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:14.464730 30400 heartbeater.cc:511] Master 127.15.217.126:38731 requested a full tablet report, sending...
I20260810 07:17:14.465097 30400 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:14.465183 30400 heartbeater.cc:511] Master 127.15.217.126:38731 requested a full tablet report, sending...
W20260810 07:17:14.465479 30400 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 07:17:14.565853 30405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:14.566113 30405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:14.566159 30405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:14.567471 30405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:14.567543 30405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:14.567591 30405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.66
I20260810 07:17:14.569279 30405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:38125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.217.84:40247/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.66:46229
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.217.66
--webserver_port=36801
--tserver_master_addrs=127.15.217.126:38731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.66
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:17:14.569577 30405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:14.569842 30405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:14.576267 30413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:14.576639 30410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:14.577772 30411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:14.578646 30405 server_base.cc:1034] running on GCE node
I20260810 07:17:14.578898 30405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:14.579286 30405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:14.580497 30405 hybrid_clock.cc:648] HybridClock initialized: now 1786346234580471 us; error 42 us; skew 500 ppm
Aug 10 07:17:14 dist-test-slave-10pc krb5kdc[29089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346234, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:14.584074 30405 init.cc:377] Logged in from keytab as kudu/127.15.217.66@KRBTEST.COM (short username kudu)
I20260810 07:17:14.584740 30405 webserver.cc:460] Webserver started at http://127.15.217.66:36801/ using document root <none> and password file <none>
I20260810 07:17:14.585045 30405 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:14.585143 30405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 07:17:14 dist-test-slave-10pc krb5kdc[29089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346234, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for HTTP/127.15.217.84@KRBTEST.COM
I20260810 07:17:14.601599 30405 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:17:14.602588 30422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:14.602764 30405 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:17:14.602919 30405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-1/wal
uuid: "73e7a349a5c545d986afbf96ec100697"
format_stamp: "Formatted at 2026-08-10 07:17:13 on dist-test-slave-10pc"
server_key: "6d049d9137f6c9677a194c4599cacffe"
server_key_iv: "e0d4f0f44181a6478cbf0395a03d40af"
server_key_version: "kuduclusterkey@0"
I20260810 07:17:14.603367 30405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:14.623700 30405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:14.626583 30405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:14.626788 30405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:14.627079 30405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:17:14.627449 30405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:17:14.627507 30405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:14.627550 30405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:17:14.627599 30405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:14.634135 30405 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.66:46229
I20260810 07:17:14.634725 30405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
I20260810 07:17:14.636688 30544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.66:46229 every 8 connection(s)
Aug 10 07:17:14 dist-test-slave-10pc krb5kdc[29089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346234, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:17:14.643977 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 30405
I20260810 07:17:14.644660 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.217.84:40247/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.67:46103
--local_ip_for_outbound_sockets=127.15.217.67
--tserver_master_addrs=127.15.217.126:38731
--webserver_port=45187
--webserver_interface=127.15.217.67
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:38125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.217.84:40247/kms with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:17:14.647841 30404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:14.635682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:56593 (local address 127.15.217.126:38731)
0810 07:17:14.635736 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:14.635738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:14.636006 (+ 268us) server_negotiation.cc:407] Connection header received
0810 07:17:14.636828 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:14.636830 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:14.636843 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:14.636876 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:14.637780 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:14.638575 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:14.640122 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:14.640422 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:14.640498 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:14.643925 (+ 3427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:14.643927 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:14.643927 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:14.643936 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:14.646196 (+ 2260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:14.646748 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:14.646749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:14.646750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:14.646822 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:14.647095 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:14.647096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:14.647097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:14.647266 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:14.647345 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:14.647654 (+ 309us) server_negotiation.cc:299] Negotiation successful
0810 07:17:14.647746 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 07:17:14.648033 30547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:14.635671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:38731 (local address 127.15.217.66:56593)
0810 07:17:14.635861 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:14.635912 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:14.636684 (+ 772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:14.637076 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:14.637081 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:14.637345 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:14.637653 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:14.637661 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:14.638705 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:14.638706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:14.640003 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:14.640007 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:14.640618 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:14.640619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:14.640866 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:14.642051 (+ 1185us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:14.642062 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:14.643803 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:14.646289 (+ 2486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:14.646296 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:14.646299 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:14.646648 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:14.646899 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:14.646900 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:14.646900 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:14.647016 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:14.647335 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:14.647342 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:14.647512 (+ 170us) client_negotiation.cc:769] Sending connection context
0810 07:17:14.647631 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 07:17:14.647788 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":103,"threads_started":1}
I20260810 07:17:14.648548 30545 heartbeater.cc:347] Connected to a master server at 127.15.217.126:38731
I20260810 07:17:14.648675 30545 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:14.648887 30545 heartbeater.cc:511] Master 127.15.217.126:38731 requested a full tablet report, sending...
W20260810 07:17:14.649791 30545 heartbeater.cc:644] Failed to heartbeat to 127.15.217.126:38731 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 07:17:14.649879 30545 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:14.649955 30545 heartbeater.cc:511] Master 127.15.217.126:38731 requested a full tablet report, sending...
I20260810 07:17:14.650339 30545 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:14.650426 30545 heartbeater.cc:511] Master 127.15.217.126:38731 requested a full tablet report, sending...
W20260810 07:17:14.650761 30545 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 07:17:14.768577 30548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:14.768877 30548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:14.768921 30548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:14.770717 30548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:14.770790 30548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:14.770840 30548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.67
I20260810 07:17:14.773173 30548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:38125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.217.84:40247/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.67:46103
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.217.67
--webserver_port=45187
--tserver_master_addrs=127.15.217.126:38731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.67
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:17:14.773490 30548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:14.773790 30548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:14.780325 30554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:14.780339 30556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:14.780336 30558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:14.780767 30548 server_base.cc:1034] running on GCE node
I20260810 07:17:14.781085 30548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:14.781376 30548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:14.782531 30548 hybrid_clock.cc:648] HybridClock initialized: now 1786346234782499 us; error 47 us; skew 500 ppm
Aug 10 07:17:14 dist-test-slave-10pc krb5kdc[29089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346234, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:14.785599 30548 init.cc:377] Logged in from keytab as kudu/127.15.217.67@KRBTEST.COM (short username kudu)
I20260810 07:17:14.786136 30548 webserver.cc:460] Webserver started at http://127.15.217.67:45187/ using document root <none> and password file <none>
I20260810 07:17:14.786405 30548 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:14.786499 30548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 07:17:14 dist-test-slave-10pc krb5kdc[29089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346234, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for HTTP/127.15.217.84@KRBTEST.COM
I20260810 07:17:14.802148 30548 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:17:14.810343 30567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:14.810616 30548 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.001s sys 0.000s
I20260810 07:17:14.810712 30548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-2/wal
uuid: "4048b33da76d4ae9829eabf5eb25a9fb"
format_stamp: "Formatted at 2026-08-10 07:17:13 on dist-test-slave-10pc"
server_key: "86252225d7d31bc7b619b5b322bcae26"
server_key_iv: "13c5be2a78545d3150d99a7edb68a585"
server_key_version: "kuduclusterkey@0"
I20260810 07:17:14.811259 30548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:14.827545 30548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:14.830179 30548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:14.830370 30548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:14.830675 30548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:17:14.831092 30548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:17:14.831148 30548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:14.831187 30548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:17:14.831207 30548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:14.837742 30548 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.67:46103
I20260810 07:17:14.838229 30548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
I20260810 07:17:14.841028 30692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.67:46103 every 8 connection(s)
I20260810 07:17:14.846448 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 30548
Aug 10 07:17:14 dist-test-slave-10pc krb5kdc[29089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346234, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:17:14.852836 30404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:14.841671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:54171 (local address 127.15.217.126:38731)
0810 07:17:14.841725 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:14.841727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:14.841972 (+ 245us) server_negotiation.cc:407] Connection header received
0810 07:17:14.842743 (+ 771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:14.842745 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:14.842759 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:14.842810 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:14.843578 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:14.844335 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:14.845808 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:14.846106 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:14.846175 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:14.849538 (+ 3363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:14.849540 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:14.849540 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:14.849549 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:14.851244 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:14.851716 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:14.851717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:14.851717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:14.851782 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:14.852074 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:14.852074 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:14.852075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:14.852199 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:14.852256 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:14.852658 (+ 402us) server_negotiation.cc:299] Negotiation successful
0810 07:17:14.852743 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 07:17:14.853056 30696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:14.841686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:38731 (local address 127.15.217.67:54171)
0810 07:17:14.841840 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:14.841884 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:14.842577 (+ 693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:14.842887 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:14.842892 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:14.843161 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:14.843451 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:14.843458 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:14.844501 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:14.844502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:14.845698 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:14.845701 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:14.846810 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:14.846811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:14.847165 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:14.847760 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:14.847770 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:14.849438 (+ 1668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:14.851320 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:14.851327 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:14.851330 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:14.851627 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:14.851851 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:14.851851 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:14.851852 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:14.852003 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:14.852283 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:14.852289 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:14.852497 (+ 208us) client_negotiation.cc:769] Sending connection context
0810 07:17:14.852634 (+ 137us) client_negotiation.cc:240] Negotiation successful
0810 07:17:14.852790 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":84,"threads_started":1}
I20260810 07:17:14.853605 30694 heartbeater.cc:347] Connected to a master server at 127.15.217.126:38731
I20260810 07:17:14.853708 30694 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:14.853902 30694 heartbeater.cc:511] Master 127.15.217.126:38731 requested a full tablet report, sending...
W20260810 07:17:14.854579 30694 heartbeater.cc:644] Failed to heartbeat to 127.15.217.126:38731 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 07:17:14.854665 30694 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:14.854743 30694 heartbeater.cc:511] Master 127.15.217.126:38731 requested a full tablet report, sending...
I20260810 07:17:14.855297 30694 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:14.855382 30694 heartbeater.cc:511] Master 127.15.217.126:38731 requested a full tablet report, sending...
W20260810 07:17:14.855695 30694 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 07:17:14.864727 30699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:14.846913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50160 (local address 127.15.217.126:38731)
0810 07:17:14.847430 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:14.847441 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:14.847605 (+ 164us) server_negotiation.cc:407] Connection header received
0810 07:17:14.848537 (+ 932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:14.848538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:14.848553 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:14.848623 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:14.849556 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:14.851009 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:14.853351 (+ 2342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:14.853655 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:14.853716 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:14.856372 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:14.856373 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:14.856374 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:14.856381 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:14.858199 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:14.863452 (+ 5253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:14.863453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:14.863454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:14.863519 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:14.863725 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:14.863726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:14.863726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:14.863863 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:14.864055 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:14.864524 (+ 469us) server_negotiation.cc:299] Negotiation successful
0810 07:17:14.864620 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":88,"threads_started":1}
Aug 10 07:17:15 dist-test-slave-10pc krb5kdc[29089](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786346235, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:15.465699 30400 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:15.465894 30400 heartbeater.cc:511] Master 127.15.217.126:38731 requested a full tablet report, sending...
I20260810 07:17:15.466394 30184 ts_manager.cc:194] Registered new tserver with Master: 40c1e67b940a4439ba04220a0d7f1e90 (127.15.217.65:43377)
I20260810 07:17:15.467190 30184 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.65@KRBTEST.COM'} at 127.15.217.65:39559
I20260810 07:17:15.650947 30545 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:15.651124 30545 heartbeater.cc:511] Master 127.15.217.126:38731 requested a full tablet report, sending...
I20260810 07:17:15.651547 30184 ts_manager.cc:194] Registered new tserver with Master: 73e7a349a5c545d986afbf96ec100697 (127.15.217.66:46229)
I20260810 07:17:15.651963 30184 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.66@KRBTEST.COM'} at 127.15.217.66:56593
Aug 10 07:17:15 dist-test-slave-10pc krb5kdc[29089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346235, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for HTTP/127.15.217.84@KRBTEST.COM
I20260810 07:17:15.855901 30694 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:15.856076 30694 heartbeater.cc:511] Master 127.15.217.126:38731 requested a full tablet report, sending...
I20260810 07:17:15.856343 30184 ts_manager.cc:194] Registered new tserver with Master: 4048b33da76d4ae9829eabf5eb25a9fb (127.15.217.67:46103)
I20260810 07:17:15.856913 30184 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.67@KRBTEST.COM'} at 127.15.217.67:54171
I20260810 07:17:15.866403 16229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:17:15.874601 30719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:15.867439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50176 (local address 127.15.217.126:38731)
0810 07:17:15.867554 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:15.867556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:15.867713 (+ 157us) server_negotiation.cc:407] Connection header received
0810 07:17:15.867782 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:15.867783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:15.867796 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:15.867861 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:15.868436 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:15.868981 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:15.869718 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:15.869944 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:15.870021 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:15.871314 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:15.871315 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:15.871316 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:15.871323 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:15.873148 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:15.873602 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:15.873604 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:15.873604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:15.873676 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:15.873906 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:15.873907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:15.873907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:15.874047 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:15.874153 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:15.874307 (+ 154us) server_negotiation.cc:299] Negotiation successful
0810 07:17:15.874395 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":64,"threads_started":1}
I20260810 07:17:15.876955 30184 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50176:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:17:15.879182 30184 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:17:15.887601 30728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:15.882459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35362 (local address 127.15.217.67:46103)
0810 07:17:15.882743 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:15.882749 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:15.883215 (+ 466us) server_negotiation.cc:407] Connection header received
0810 07:17:15.883316 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:15.883318 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:15.883335 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:15.883386 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:17:15.883657 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:15.884776 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:15.886303 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:15.887067 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:15.887122 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:15.887176 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:15.887450 (+ 274us) server_negotiation.cc:299] Negotiation successful
0810 07:17:15.887512 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":53,"threads_started":1}
I20260810 07:17:15.887770 30726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:15.882130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:43377 (local address 127.0.0.1:50178)
0810 07:17:15.882840 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:15.882882 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:15.882957 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:15.883482 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:15.883486 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:15.883489 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:17:15.883570 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:15.883572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:15.884805 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:15.884805 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:15.886167 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:15.886171 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:15.887419 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:15.887419 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:15.887488 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:15.887492 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:17:15.887534 (+ 42us) client_negotiation.cc:240] Negotiation successful
0810 07:17:15.887693 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"thread_start_us":86,"threads_started":1}
I20260810 07:17:15.888119 30729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:15.882566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50178 (local address 127.15.217.65:43377)
0810 07:17:15.882731 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:15.882736 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:15.883224 (+ 488us) server_negotiation.cc:407] Connection header received
0810 07:17:15.883265 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:15.883267 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:15.883284 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:15.883358 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:17:15.883653 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:15.884708 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:15.886272 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:15.886984 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:15.887058 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:15.887957 (+ 899us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:15.887984 (+ 27us) server_negotiation.cc:299] Negotiation successful
0810 07:17:15.888029 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":73,"threads_started":1}
I20260810 07:17:15.888134 30724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:15.882356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:46103 (local address 127.0.0.1:35362)
0810 07:17:15.883165 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:15.883172 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:15.883238 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:15.883500 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:15.883500 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:15.883503 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:17:15.883576 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:15.883578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:15.884880 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:15.884881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:15.886194 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:15.886197 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:15.887241 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:15.887242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:15.887329 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:15.887343 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 07:17:15.888017 (+ 674us) client_negotiation.cc:240] Negotiation successful
0810 07:17:15.888065 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":797,"thread_start_us":69,"threads_started":1}
I20260810 07:17:15.888875 30326 tablet_service.cc:1467] Processing CreateTablet for tablet a5bfd3b58b504280b910cc0feea1cac5 (DEFAULT_TABLE table=test-table [id=20a17f85fe5f412b95b41c02bf805170]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:17:15.889045 30622 tablet_service.cc:1467] Processing CreateTablet for tablet a5bfd3b58b504280b910cc0feea1cac5 (DEFAULT_TABLE table=test-table [id=20a17f85fe5f412b95b41c02bf805170]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:17:15.889298 30326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5bfd3b58b504280b910cc0feea1cac5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:17:15.889421 30622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5bfd3b58b504280b910cc0feea1cac5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:17:15.892560 30725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:15.882177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:46229 (local address 127.0.0.1:52900)
0810 07:17:15.883056 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:15.883064 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:15.883099 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:15.888745 (+ 5646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:15.888746 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:15.888749 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:17:15.888789 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:15.888790 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:15.890201 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:15.890202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:15.891239 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:15.891242 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:15.892269 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:15.892270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:15.892340 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:15.892345 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:17:15.892397 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 07:17:15.892435 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":866,"thread_start_us":80,"threads_started":1}
I20260810 07:17:15.893137 30727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:15.882400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52900 (local address 127.15.217.66:46229)
0810 07:17:15.888547 (+ 6147us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:15.888552 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:15.888563 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:17:15.888601 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:15.888603 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:15.888621 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:15.888740 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:17:15.888842 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:15.889831 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:15.891345 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:15.892097 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:15.892177 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:15.892231 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:15.893003 (+ 772us) server_negotiation.cc:299] Negotiation successful
0810 07:17:15.893050 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6107,"thread_start_us":50,"threads_started":1}
I20260810 07:17:15.894191 30473 tablet_service.cc:1467] Processing CreateTablet for tablet a5bfd3b58b504280b910cc0feea1cac5 (DEFAULT_TABLE table=test-table [id=20a17f85fe5f412b95b41c02bf805170]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:17:15.894546 30473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5bfd3b58b504280b910cc0feea1cac5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:17:15.899305 30732 tablet_bootstrap.cc:492] T a5bfd3b58b504280b910cc0feea1cac5 P 4048b33da76d4ae9829eabf5eb25a9fb: Bootstrap starting.
I20260810 07:17:15.900063 30732 tablet_bootstrap.cc:654] T a5bfd3b58b504280b910cc0feea1cac5 P 4048b33da76d4ae9829eabf5eb25a9fb: Neither blocks nor log segments found. Creating new log.
I20260810 07:17:15.900063 30730 tablet_bootstrap.cc:492] T a5bfd3b58b504280b910cc0feea1cac5 P 40c1e67b940a4439ba04220a0d7f1e90: Bootstrap starting.
I20260810 07:17:15.900427 30732 log.cc:824] T a5bfd3b58b504280b910cc0feea1cac5 P 4048b33da76d4ae9829eabf5eb25a9fb: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:15.900682 30733 tablet_bootstrap.cc:492] T a5bfd3b58b504280b910cc0feea1cac5 P 73e7a349a5c545d986afbf96ec100697: Bootstrap starting.
I20260810 07:17:15.900768 30730 tablet_bootstrap.cc:654] T a5bfd3b58b504280b910cc0feea1cac5 P 40c1e67b940a4439ba04220a0d7f1e90: Neither blocks nor log segments found. Creating new log.
I20260810 07:17:15.901114 30730 log.cc:824] T a5bfd3b58b504280b910cc0feea1cac5 P 40c1e67b940a4439ba04220a0d7f1e90: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:15.901347 30732 tablet_bootstrap.cc:492] T a5bfd3b58b504280b910cc0feea1cac5 P 4048b33da76d4ae9829eabf5eb25a9fb: No bootstrap required, opened a new log
I20260810 07:17:15.901450 30733 tablet_bootstrap.cc:654] T a5bfd3b58b504280b910cc0feea1cac5 P 73e7a349a5c545d986afbf96ec100697: Neither blocks nor log segments found. Creating new log.
I20260810 07:17:15.901453 30732 ts_tablet_manager.cc:1397] T a5bfd3b58b504280b910cc0feea1cac5 P 4048b33da76d4ae9829eabf5eb25a9fb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:17:15.901754 30733 log.cc:824] T a5bfd3b58b504280b910cc0feea1cac5 P 73e7a349a5c545d986afbf96ec100697: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:15.902359 30730 tablet_bootstrap.cc:492] T a5bfd3b58b504280b910cc0feea1cac5 P 40c1e67b940a4439ba04220a0d7f1e90: No bootstrap required, opened a new log
I20260810 07:17:15.902436 30730 ts_tablet_manager.cc:1397] T a5bfd3b58b504280b910cc0feea1cac5 P 40c1e67b940a4439ba04220a0d7f1e90: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:17:15.904321 30732 raft_consensus.cc:348] T a5bfd3b58b504280b910cc0feea1cac5 P 4048b33da76d4ae9829eabf5eb25a9fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40c1e67b940a4439ba04220a0d7f1e90" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 43377 } } peers { permanent_uuid: "4048b33da76d4ae9829eabf5eb25a9fb" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 46103 } } peers { permanent_uuid: "73e7a349a5c545d986afbf96ec100697" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46229 } }
I20260810 07:17:15.904549 30732 raft_consensus.cc:374] T a5bfd3b58b504280b910cc0feea1cac5 P 4048b33da76d4ae9829eabf5eb25a9fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:17:15.904598 30732 raft_consensus.cc:729] T a5bfd3b58b504280b910cc0feea1cac5 P 4048b33da76d4ae9829eabf5eb25a9fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4048b33da76d4ae9829eabf5eb25a9fb, State: Initialized, Role: FOLLOWER
I20260810 07:17:15.904728 30732 consensus_queue.cc:260] T a5bfd3b58b504280b910cc0feea1cac5 P 4048b33da76d4ae9829eabf5eb25a9fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40c1e67b940a4439ba04220a0d7f1e90" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 43377 } } peers { permanent_uuid: "4048b33da76d4ae9829eabf5eb25a9fb" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 46103 } } peers { permanent_uuid: "73e7a349a5c545d986afbf96ec100697" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46229 } }
I20260810 07:17:15.905052 30694 heartbeater.cc:503] Master 127.15.217.126:38731 was elected leader, sending a full tablet report...
I20260810 07:17:15.905081 30730 raft_consensus.cc:348] T a5bfd3b58b504280b910cc0feea1cac5 P 40c1e67b940a4439ba04220a0d7f1e90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40c1e67b940a4439ba04220a0d7f1e90" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 43377 } } peers { permanent_uuid: "4048b33da76d4ae9829eabf5eb25a9fb" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 46103 } } peers { permanent_uuid: "73e7a349a5c545d986afbf96ec100697" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46229 } }
I20260810 07:17:15.905253 30730 raft_consensus.cc:374] T a5bfd3b58b504280b910cc0feea1cac5 P 40c1e67b940a4439ba04220a0d7f1e90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:17:15.905304 30730 raft_consensus.cc:729] T a5bfd3b58b504280b910cc0feea1cac5 P 40c1e67b940a4439ba04220a0d7f1e90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40c1e67b940a4439ba04220a0d7f1e90, State: Initialized, Role: FOLLOWER
I20260810 07:17:15.905035 30732 ts_tablet_manager.cc:1428] T a5bfd3b58b504280b910cc0feea1cac5 P 4048b33da76d4ae9829eabf5eb25a9fb: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260810 07:17:15.905450 30730 consensus_queue.cc:260] T a5bfd3b58b504280b910cc0feea1cac5 P 40c1e67b940a4439ba04220a0d7f1e90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40c1e67b940a4439ba04220a0d7f1e90" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 43377 } } peers { permanent_uuid: "4048b33da76d4ae9829eabf5eb25a9fb" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 46103 } } peers { permanent_uuid: "73e7a349a5c545d986afbf96ec100697" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46229 } }
I20260810 07:17:15.905681 30733 tablet_bootstrap.cc:492] T a5bfd3b58b504280b910cc0feea1cac5 P 73e7a349a5c545d986afbf96ec100697: No bootstrap required, opened a new log
I20260810 07:17:15.905741 30733 ts_tablet_manager.cc:1397] T a5bfd3b58b504280b910cc0feea1cac5 P 73e7a349a5c545d986afbf96ec100697: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260810 07:17:15.905758 30400 heartbeater.cc:503] Master 127.15.217.126:38731 was elected leader, sending a full tablet report...
I20260810 07:17:15.905982 30730 ts_tablet_manager.cc:1428] T a5bfd3b58b504280b910cc0feea1cac5 P 40c1e67b940a4439ba04220a0d7f1e90: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260810 07:17:15.907982 30733 raft_consensus.cc:348] T a5bfd3b58b504280b910cc0feea1cac5 P 73e7a349a5c545d986afbf96ec100697 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40c1e67b940a4439ba04220a0d7f1e90" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 43377 } } peers { permanent_uuid: "4048b33da76d4ae9829eabf5eb25a9fb" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 46103 } } peers { permanent_uuid: "73e7a349a5c545d986afbf96ec100697" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46229 } }
I20260810 07:17:15.908154 30733 raft_consensus.cc:374] T a5bfd3b58b504280b910cc0feea1cac5 P 73e7a349a5c545d986afbf96ec100697 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:17:15.908187 30733 raft_consensus.cc:729] T a5bfd3b58b504280b910cc0feea1cac5 P 73e7a349a5c545d986afbf96ec100697 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73e7a349a5c545d986afbf96ec100697, State: Initialized, Role: FOLLOWER
I20260810 07:17:15.908294 30733 consensus_queue.cc:260] T a5bfd3b58b504280b910cc0feea1cac5 P 73e7a349a5c545d986afbf96ec100697 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40c1e67b940a4439ba04220a0d7f1e90" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 43377 } } peers { permanent_uuid: "4048b33da76d4ae9829eabf5eb25a9fb" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 46103 } } peers { permanent_uuid: "73e7a349a5c545d986afbf96ec100697" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46229 } }
I20260810 07:17:15.908560 30733 ts_tablet_manager.cc:1428] T a5bfd3b58b504280b910cc0feea1cac5 P 73e7a349a5c545d986afbf96ec100697: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260810 07:17:15.908607 30545 heartbeater.cc:503] Master 127.15.217.126:38731 was elected leader, sending a full tablet report...
W20260810 07:17:15.946858 30401 tablet.cc:2369] T a5bfd3b58b504280b910cc0feea1cac5 P 40c1e67b940a4439ba04220a0d7f1e90: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 07:17:16 dist-test-slave-10pc krb5kdc[29089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346235, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for HTTP/127.15.217.84@KRBTEST.COM
W20260810 07:17:16.092517 30695 tablet.cc:2369] T a5bfd3b58b504280b910cc0feea1cac5 P 4048b33da76d4ae9829eabf5eb25a9fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:17:16.115139 30740 raft_consensus.cc:482] T a5bfd3b58b504280b910cc0feea1cac5 P 73e7a349a5c545d986afbf96ec100697 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:17:16.115322 30740 raft_consensus.cc:504] T a5bfd3b58b504280b910cc0feea1cac5 P 73e7a349a5c545d986afbf96ec100697 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40c1e67b940a4439ba04220a0d7f1e90" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 43377 } } peers { permanent_uuid: "4048b33da76d4ae9829eabf5eb25a9fb" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 46103 } } peers { permanent_uuid: "73e7a349a5c545d986afbf96ec100697" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46229 } }
I20260810 07:17:16.115794 30740 leader_election.cc:288] T a5bfd3b58b504280b910cc0feea1cac5 P 73e7a349a5c545d986afbf96ec100697 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 40c1e67b940a4439ba04220a0d7f1e90 (127.15.217.65:43377), 4048b33da76d4ae9829eabf5eb25a9fb (127.15.217.67:46103)
I20260810 07:17:16.122558 30729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:16.116417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:53135 (local address 127.15.217.65:43377)
0810 07:17:16.116495 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:16.116497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:16.117073 (+ 576us) server_negotiation.cc:407] Connection header received
0810 07:17:16.117185 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:16.117186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:16.117197 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:16.117228 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:17:16.117468 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:16.118255 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:16.121176 (+ 2921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:16.121763 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:16.121817 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:16.121829 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:16.122417 (+ 588us) server_negotiation.cc:299] Negotiation successful
0810 07:17:16.122476 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 07:17:16.123190 30746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:16.115992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:43377 (local address 127.15.217.66:53135)
0810 07:17:16.116999 (+ 1007us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:16.117014 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:16.117106 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:16.117301 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:16.117302 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:16.117304 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:17:16.117390 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:16.117392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:16.118797 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:16.118797 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:16.121063 (+ 2266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:16.121068 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:16.122245 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:16.122246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:16.122321 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:16.122326 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:17:16.123029 (+ 703us) client_negotiation.cc:240] Negotiation successful
0810 07:17:16.123082 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":985,"thread_start_us":118,"threads_started":1}
I20260810 07:17:16.123736 30347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5bfd3b58b504280b910cc0feea1cac5" candidate_uuid: "73e7a349a5c545d986afbf96ec100697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40c1e67b940a4439ba04220a0d7f1e90" is_pre_election: true
I20260810 07:17:16.123925 30347 raft_consensus.cc:2454] T a5bfd3b58b504280b910cc0feea1cac5 P 40c1e67b940a4439ba04220a0d7f1e90 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73e7a349a5c545d986afbf96ec100697 in term 0.
I20260810 07:17:16.124310 30426 leader_election.cc:302] T a5bfd3b58b504280b910cc0feea1cac5 P 73e7a349a5c545d986afbf96ec100697 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40c1e67b940a4439ba04220a0d7f1e90, 73e7a349a5c545d986afbf96ec100697; no voters:
I20260810 07:17:16.124475 30740 raft_consensus.cc:2781] T a5bfd3b58b504280b910cc0feea1cac5 P 73e7a349a5c545d986afbf96ec100697 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:17:16.124528 30740 raft_consensus.cc:482] T a5bfd3b58b504280b910cc0feea1cac5 P 73e7a349a5c545d986afbf96ec100697 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:17:16.124552 30740 raft_consensus.cc:3037] T a5bfd3b58b504280b910cc0feea1cac5 P 73e7a349a5c545d986afbf96ec100697 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:17:16.124902 30728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:16.116546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:42155 (local address 127.15.217.67:46103)
0810 07:17:16.116591 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:16.116593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:16.120664 (+ 4071us) server_negotiation.cc:407] Connection header received
0810 07:17:16.120898 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:16.120899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:16.120910 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:16.120939 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:17:16.121712 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:16.122394 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:16.123919 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:16.124509 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:16.124563 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:16.124572 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:16.124791 (+ 219us) server_negotiation.cc:299] Negotiation successful
0810 07:17:16.124834 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 07:17:16.125170 30744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:16.116221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:46103 (local address 127.15.217.66:42155)
0810 07:17:16.120547 (+ 4326us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:16.120560 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:16.120807 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:16.121548 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:16.121549 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:16.121552 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:17:16.121627 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:16.121629 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:16.122754 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:16.122754 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:16.123806 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:16.123810 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:16.124638 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:16.124639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:16.124711 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:16.124714 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:17:16.125044 (+ 330us) client_negotiation.cc:240] Negotiation successful
0810 07:17:16.125090 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4305,"thread_start_us":62,"threads_started":1}
I20260810 07:17:16.125568 30740 raft_consensus.cc:504] T a5bfd3b58b504280b910cc0feea1cac5 P 73e7a349a5c545d986afbf96ec100697 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40c1e67b940a4439ba04220a0d7f1e90" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 43377 } } peers { permanent_uuid: "4048b33da76d4ae9829eabf5eb25a9fb" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 46103 } } peers { permanent_uuid: "73e7a349a5c545d986afbf96ec100697" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46229 } }
I20260810 07:17:16.125694 30740 leader_election.cc:288] T a5bfd3b58b504280b910cc0feea1cac5 P 73e7a349a5c545d986afbf96ec100697 [CANDIDATE]: Term 1 election: Requested vote from peers 40c1e67b940a4439ba04220a0d7f1e90 (127.15.217.65:43377), 4048b33da76d4ae9829eabf5eb25a9fb (127.15.217.67:46103)
I20260810 07:17:16.126315 30347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5bfd3b58b504280b910cc0feea1cac5" candidate_uuid: "73e7a349a5c545d986afbf96ec100697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40c1e67b940a4439ba04220a0d7f1e90"
I20260810 07:17:16.126405 30347 raft_consensus.cc:3037] T a5bfd3b58b504280b910cc0feea1cac5 P 40c1e67b940a4439ba04220a0d7f1e90 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:17:16.126446 30639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5bfd3b58b504280b910cc0feea1cac5" candidate_uuid: "73e7a349a5c545d986afbf96ec100697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4048b33da76d4ae9829eabf5eb25a9fb" is_pre_election: true
I20260810 07:17:16.126601 30639 raft_consensus.cc:2454] T a5bfd3b58b504280b910cc0feea1cac5 P 4048b33da76d4ae9829eabf5eb25a9fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73e7a349a5c545d986afbf96ec100697 in term 0.
I20260810 07:17:16.126752 30638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5bfd3b58b504280b910cc0feea1cac5" candidate_uuid: "73e7a349a5c545d986afbf96ec100697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4048b33da76d4ae9829eabf5eb25a9fb"
I20260810 07:17:16.126816 30638 raft_consensus.cc:3037] T a5bfd3b58b504280b910cc0feea1cac5 P 4048b33da76d4ae9829eabf5eb25a9fb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:17:16.127401 30347 raft_consensus.cc:2454] T a5bfd3b58b504280b910cc0feea1cac5 P 40c1e67b940a4439ba04220a0d7f1e90 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73e7a349a5c545d986afbf96ec100697 in term 1.
I20260810 07:17:16.127573 30426 leader_election.cc:302] T a5bfd3b58b504280b910cc0feea1cac5 P 73e7a349a5c545d986afbf96ec100697 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40c1e67b940a4439ba04220a0d7f1e90, 73e7a349a5c545d986afbf96ec100697; no voters:
I20260810 07:17:16.127707 30740 raft_consensus.cc:2781] T a5bfd3b58b504280b910cc0feea1cac5 P 73e7a349a5c545d986afbf96ec100697 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:17:16.127835 30638 raft_consensus.cc:2454] T a5bfd3b58b504280b910cc0feea1cac5 P 4048b33da76d4ae9829eabf5eb25a9fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73e7a349a5c545d986afbf96ec100697 in term 1.
I20260810 07:17:16.127878 30740 raft_consensus.cc:686] T a5bfd3b58b504280b910cc0feea1cac5 P 73e7a349a5c545d986afbf96ec100697 [term 1 LEADER]: Becoming Leader. State: Replica: 73e7a349a5c545d986afbf96ec100697, State: Running, Role: LEADER
I20260810 07:17:16.127986 30740 consensus_queue.cc:237] T a5bfd3b58b504280b910cc0feea1cac5 P 73e7a349a5c545d986afbf96ec100697 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40c1e67b940a4439ba04220a0d7f1e90" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 43377 } } peers { permanent_uuid: "4048b33da76d4ae9829eabf5eb25a9fb" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 46103 } } peers { permanent_uuid: "73e7a349a5c545d986afbf96ec100697" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46229 } }
I20260810 07:17:16.128978 30183 catalog_manager.cc:5495] T a5bfd3b58b504280b910cc0feea1cac5 P 73e7a349a5c545d986afbf96ec100697 reported cstate change: term changed from 0 to 1, leader changed from <none> to 73e7a349a5c545d986afbf96ec100697 (127.15.217.66). New cstate: current_term: 1 leader_uuid: "73e7a349a5c545d986afbf96ec100697" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40c1e67b940a4439ba04220a0d7f1e90" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4048b33da76d4ae9829eabf5eb25a9fb" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 46103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73e7a349a5c545d986afbf96ec100697" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46229 } health_report { overall_health: HEALTHY } } }
W20260810 07:17:16.137912 30546 tablet.cc:2369] T a5bfd3b58b504280b910cc0feea1cac5 P 73e7a349a5c545d986afbf96ec100697: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:17:16.169771 30727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:16.166298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52904 (local address 127.15.217.66:46229)
0810 07:17:16.166344 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:16.166346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:16.166354 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:17:16.166397 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:16.166398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:16.166419 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:16.166484 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:17:16.166707 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:16.167474 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:16.168660 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:16.168944 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:16.169017 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:16.169231 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:17:16.169375 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:17:16.169448 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:16.169668 (+ 220us) server_negotiation.cc:299] Negotiation successful
0810 07:17:16.169692 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 07:17:16.176297 30347 raft_consensus.cc:1261] T a5bfd3b58b504280b910cc0feea1cac5 P 40c1e67b940a4439ba04220a0d7f1e90 [term 1 FOLLOWER]: Refusing update from remote peer 73e7a349a5c545d986afbf96ec100697: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:17:16.176297 30638 raft_consensus.cc:1261] T a5bfd3b58b504280b910cc0feea1cac5 P 4048b33da76d4ae9829eabf5eb25a9fb [term 1 FOLLOWER]: Refusing update from remote peer 73e7a349a5c545d986afbf96ec100697: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:17:16.176834 30740 consensus_queue.cc:1035] T a5bfd3b58b504280b910cc0feea1cac5 P 73e7a349a5c545d986afbf96ec100697 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4048b33da76d4ae9829eabf5eb25a9fb" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 46103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:17:16.177093 30747 consensus_queue.cc:1035] T a5bfd3b58b504280b910cc0feea1cac5 P 73e7a349a5c545d986afbf96ec100697 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40c1e67b940a4439ba04220a0d7f1e90" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:17:16.184537 30750 mvcc.cc:205] Tried to move back new op lower bound from 7316874183374237696 to 7316874183180955648. Current Snapshot: MvccSnapshot[applied={T|T < 7316874183374237696}]
I20260810 07:17:16.185111 30753 mvcc.cc:205] Tried to move back new op lower bound from 7316874183374237696 to 7316874183180955648. Current Snapshot: MvccSnapshot[applied={T|T < 7316874183374237696}]
I20260810 07:17:16.185724 30751 mvcc.cc:205] Tried to move back new op lower bound from 7316874183374237696 to 7316874183180955648. Current Snapshot: MvccSnapshot[applied={T|T < 7316874183374237696}]
I20260810 07:17:16.188709 30183 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50176:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:17:16.188849 30183 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50176:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:17:16.189610 30183 catalog_manager.cc:5782] T 00000000000000000000000000000000 P db37adc733684cc1aa5a0dcf1794eca0: Sending DeleteTablet for 3 replicas of tablet a5bfd3b58b504280b910cc0feea1cac5
I20260810 07:17:16.190095 30326 tablet_service.cc:1514] Processing DeleteTablet for tablet a5bfd3b58b504280b910cc0feea1cac5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:17:16 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:50178
I20260810 07:17:16.190337 30622 tablet_service.cc:1514] Processing DeleteTablet for tablet a5bfd3b58b504280b910cc0feea1cac5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:17:16 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:35362
I20260810 07:17:16.190697 30473 tablet_service.cc:1514] Processing DeleteTablet for tablet a5bfd3b58b504280b910cc0feea1cac5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:17:16 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:52900
I20260810 07:17:16.191241 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 30224
I20260810 07:17:16.207880 30760 tablet_replica.cc:330] T a5bfd3b58b504280b910cc0feea1cac5 P 4048b33da76d4ae9829eabf5eb25a9fb: stopping tablet replica
I20260810 07:17:16.208175 30760 raft_consensus.cc:2229] T a5bfd3b58b504280b910cc0feea1cac5 P 4048b33da76d4ae9829eabf5eb25a9fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:17:16.208464 30760 raft_consensus.cc:2258] T a5bfd3b58b504280b910cc0feea1cac5 P 4048b33da76d4ae9829eabf5eb25a9fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:17:16.208757 30761 tablet_replica.cc:330] T a5bfd3b58b504280b910cc0feea1cac5 P 73e7a349a5c545d986afbf96ec100697: stopping tablet replica
I20260810 07:17:16.209244 30760 ts_tablet_manager.cc:1905] T a5bfd3b58b504280b910cc0feea1cac5 P 4048b33da76d4ae9829eabf5eb25a9fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:17:16.211099 30761 raft_consensus.cc:2229] T a5bfd3b58b504280b910cc0feea1cac5 P 73e7a349a5c545d986afbf96ec100697 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:17:16.211337 30761 raft_consensus.cc:2258] T a5bfd3b58b504280b910cc0feea1cac5 P 73e7a349a5c545d986afbf96ec100697 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:17:16.211576 30760 ts_tablet_manager.cc:1918] T a5bfd3b58b504280b910cc0feea1cac5 P 4048b33da76d4ae9829eabf5eb25a9fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:17:16.212064 30761 ts_tablet_manager.cc:1905] T a5bfd3b58b504280b910cc0feea1cac5 P 73e7a349a5c545d986afbf96ec100697: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:17:16.213735 30761 ts_tablet_manager.cc:1918] T a5bfd3b58b504280b910cc0feea1cac5 P 73e7a349a5c545d986afbf96ec100697: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:17:16.213901 30760 log.cc:1196] T a5bfd3b58b504280b910cc0feea1cac5 P 4048b33da76d4ae9829eabf5eb25a9fb: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-2/wal/wals/a5bfd3b58b504280b910cc0feea1cac5
I20260810 07:17:16.214274 30760 ts_tablet_manager.cc:1939] T a5bfd3b58b504280b910cc0feea1cac5 P 4048b33da76d4ae9829eabf5eb25a9fb: Deleting consensus metadata
I20260810 07:17:16.214771 30761 log.cc:1196] T a5bfd3b58b504280b910cc0feea1cac5 P 73e7a349a5c545d986afbf96ec100697: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346081759552-16229-0/minicluster-data/ts-1/wal/wals/a5bfd3b58b504280b910cc0feea1cac5
I20260810 07:17:16.215219 30761 ts_tablet_manager.cc:1939] T a5bfd3b58b504280b910cc0feea1cac5 P 73e7a349a5c545d986afbf96ec100697: Deleting consensus metadata
I20260810 07:17:16.215473 30171 catalog_manager.cc:4841] TS 4048b33da76d4ae9829eabf5eb25a9fb (127.15.217.67:46103): tablet a5bfd3b58b504280b910cc0feea1cac5 (table test-table [id=20a17f85fe5f412b95b41c02bf805170]) successfully deleted
I20260810 07:17:16.216679 30170 catalog_manager.cc:4841] TS 73e7a349a5c545d986afbf96ec100697 (127.15.217.66:46229): tablet a5bfd3b58b504280b910cc0feea1cac5 (table test-table [id=20a17f85fe5f412b95b41c02bf805170]) successfully deleted
W20260810 07:17:16.217118 30171 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.217.65:43377 (error 108)
I20260810 07:17:16.220839 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 30405
I20260810 07:17:16.236971 30725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:16.236765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:43377 (local address 127.0.0.1:50186)
0810 07:17:16.236835 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:16.236898 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.65:43377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260810 07:17:16.237402 30171 catalog_manager.cc:4584] TS 40c1e67b940a4439ba04220a0d7f1e90 (127.15.217.65:43377): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a5bfd3b58b504280b910cc0feea1cac5: Network error: Client connection negotiation failed: client connection to 127.15.217.65:43377: connect: Connection refused (error 111)
I20260810 07:17:16.240054 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 30548
I20260810 07:17:16.250801 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 30150
I20260810 07:17:16.259727 16229 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 07:17:16.604398 16229 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 07:17:16.604550 16229 mini_ranger.cc:66] Stopping Ranger...
I20260810 07:17:16.949525 16229 mini_ranger.cc:68] Stopped Ranger
2026-08-10 07:17:16.949 UTC [29123] LOG: received smart shutdown request
2026-08-10 07:17:16.954 UTC [29123] LOG: background worker "logical replication launcher" (PID 29133) exited with exit code 1
2026-08-10 07:17:16.955 UTC [29128] LOG: shutting down
2026-08-10 07:17:17.143 UTC [29123] LOG: database system is shut down
2026-08-10T07:17:17Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49804 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:17:17 dist-test-slave-10pc krb5kdc[30772](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:17:17 dist-test-slave-10pc krb5kdc[30772](info): set up 2 sockets
Aug 10 07:17:17 dist-test-slave-10pc krb5kdc[30772](info): commencing operation
krb5kdc: starting...
W20260810 07:17:18.408100 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:17:18 dist-test-slave-10pc krb5kdc[30772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:17:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:17:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
I20260810 07:17:18.535878 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:33243
--webserver_interface=127.15.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:45739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:33243,127.15.217.125:39791,127.15.217.124:42533
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:17:18.621637 30787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:18.621876 30787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:18.621914 30787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:18.623061 30787 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:17:18.623126 30787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:18.623153 30787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:18.623184 30787 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:17:18.623211 30787 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:17:18.624794 30787 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:45739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:33243,127.15.217.125:39791,127.15.217.124:42533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:33243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:17:18.625092 30787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:18.625334 30787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:18.630322 30795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:18.630318 30793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:18.630587 30787 server_base.cc:1034] running on GCE node
W20260810 07:17:18.630384 30792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:18.630976 30787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:18.631309 30787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:18.632543 30787 hybrid_clock.cc:648] HybridClock initialized: now 1786346238632525 us; error 49 us; skew 500 ppm
Aug 10 07:17:18 dist-test-slave-10pc krb5kdc[30772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346238, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:18.639380 30787 init.cc:377] Logged in from keytab as kudu/127.15.217.126@KRBTEST.COM (short username kudu)
I20260810 07:17:18.639919 30787 webserver.cc:460] Webserver started at http://127.15.217.126:43309/ using document root <none> and password file <none>
I20260810 07:17:18.640188 30787 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:18.640261 30787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:18.640419 30787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:17:18.641577 30787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "74ddf062a4dd43b7854141f53f9b0f0b"
format_stamp: "Formatted at 2026-08-10 07:17:18 on dist-test-slave-10pc"
server_key: "5f8cfc22bc4e894711d7c1cf11de66a6"
server_key_iv: "33c109fcc9f3d816a8210e6e19592786"
server_key_version: "encryptionkey@0"
I20260810 07:17:18.641954 30787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "74ddf062a4dd43b7854141f53f9b0f0b"
format_stamp: "Formatted at 2026-08-10 07:17:18 on dist-test-slave-10pc"
server_key: "5f8cfc22bc4e894711d7c1cf11de66a6"
server_key_iv: "33c109fcc9f3d816a8210e6e19592786"
server_key_version: "encryptionkey@0"
I20260810 07:17:18.643313 30787 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:17:18.644064 30803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:18.644259 30787 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 07:17:18.644369 30787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "74ddf062a4dd43b7854141f53f9b0f0b"
format_stamp: "Formatted at 2026-08-10 07:17:18 on dist-test-slave-10pc"
server_key: "5f8cfc22bc4e894711d7c1cf11de66a6"
server_key_iv: "33c109fcc9f3d816a8210e6e19592786"
server_key_version: "encryptionkey@0"
I20260810 07:17:18.644446 30787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:18.671119 30787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:18.673434 30787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:18.673614 30787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:18.678046 30787 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:33243
I20260810 07:17:18.678088 30861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:33243 every 8 connection(s)
I20260810 07:17:18.678493 30787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:17:18.679394 30862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:17:18.681146 30862 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.217.126" port: 33243 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:17:18.681780 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 30787
I20260810 07:17:18.681912 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:17:18.682119 16229 external_mini_cluster.cc:1428] Setting key 75a6d6089664a36d3bfdebe53bf44c8c
Aug 10 07:17:18 dist-test-slave-10pc krb5kdc[30772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346238, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.217.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.125@KRBTEST.COM" created.
I20260810 07:17:18.694032 30864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:18.681922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52360 (local address 127.15.217.126:33243)
0810 07:17:18.682085 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:18.682101 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:18.684002 (+ 1901us) server_negotiation.cc:407] Connection header received
0810 07:17:18.684529 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:18.684532 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:18.684557 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:18.684668 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:18.685251 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:18.686607 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:18.687765 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:18.688101 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:18.688183 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:18.690595 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:18.690603 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:18.690603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:18.690610 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:18.692326 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:18.692772 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:18.692775 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:18.692776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:18.692862 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:18.693114 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:18.693115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:18.693115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:18.693334 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:18.693423 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:18.693621 (+ 198us) server_negotiation.cc:299] Negotiation successful
0810 07:17:18.693773 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":87,"threads_started":1}
I20260810 07:17:18.694036 30865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:18.681895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:33243 (local address 127.0.0.1:52360)
0810 07:17:18.683852 (+ 1957us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:18.683890 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:18.684347 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:18.684698 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:18.684701 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:18.684968 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:18.685140 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:18.685144 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:18.686720 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:18.686721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:18.687663 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:18.687667 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:18.688200 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:18.688202 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:18.688297 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:18.688888 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:18.688900 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:18.690489 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:18.692436 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:18.692443 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:18.692446 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:18.692671 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:18.692934 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:18.692935 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:18.692935 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:18.693032 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:18.693415 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:18.693420 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:18.693486 (+ 66us) client_negotiation.cc:769] Sending connection context
0810 07:17:18.693602 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 07:17:18.693840 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1920,"mutex_wait_us":95,"thread_start_us":109,"threads_started":1}
I20260810 07:17:18.694967 30862 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.217.125" port: 39791 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:17:18.695333 30865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:18.695220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:39791 (local address 127.0.0.1:52866)
0810 07:17:18.695268 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:18.695312 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.125:39791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260810 07:17:18.695470 30809 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.217.125:39791: connect: Connection refused (error 111)
I20260810 07:17:18.695924 30865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:18.695825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:39791 (local address 127.0.0.1:52874)
0810 07:17:18.695875 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:18.695907 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.125:39791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 07:17:18.696035 30862 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.217.125:39791: Network error: Client connection negotiation failed: client connection to 127.15.217.125:39791: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.125.keytab.
Entry for principal kudu/127.15.217.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.125.keytab.
I20260810 07:17:18.711503 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.125:39791
--webserver_interface=127.15.217.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.125.keytab
--principal=kudu/127.15.217.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:45739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:33243,127.15.217.125:39791,127.15.217.124:42533
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:17:18.745303 30862 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.217.125" port: 39791 } attempt: 1
I20260810 07:17:18.745883 30865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:18.745707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:39791 (local address 127.0.0.1:52888)
0810 07:17:18.745784 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:18.745839 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.125:39791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 07:17:18.746325 30865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:18.746220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:39791 (local address 127.0.0.1:52898)
0810 07:17:18.746267 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:18.746301 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.125:39791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260810 07:17:18.746434 30862 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.217.125:39791: Network error: Client connection negotiation failed: client connection to 127.15.217.125:39791: connect: Connection refused (error 111)
W20260810 07:17:18.798343 30869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:18.798544 30869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:18.798568 30869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:18.799715 30869 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:17:18.799758 30869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:18.799772 30869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:18.799811 30869 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:17:18.799842 30869 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:17:18.801414 30869 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:45739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:33243,127.15.217.125:39791,127.15.217.124:42533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.125.keytab
--principal=kudu/127.15.217.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.125:39791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.217.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:17:18.801676 30869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:18.801926 30869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:18.806871 30877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:18.806847 30875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:18.806864 30874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:18.807032 30869 server_base.cc:1034] running on GCE node
I20260810 07:17:18.807461 30869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:18.807710 30869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:18.808871 30869 hybrid_clock.cc:648] HybridClock initialized: now 1786346238808867 us; error 38 us; skew 500 ppm
Aug 10 07:17:18 dist-test-slave-10pc krb5kdc[30772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346238, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:18.811458 30869 init.cc:377] Logged in from keytab as kudu/127.15.217.125@KRBTEST.COM (short username kudu)
I20260810 07:17:18.811949 30869 webserver.cc:460] Webserver started at http://127.15.217.125:38769/ using document root <none> and password file <none>
I20260810 07:17:18.812204 30869 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:18.812273 30869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:18.812422 30869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:17:18.813544 30869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-1/data/instance:
uuid: "77e8a9e544c54894841ccb51383ca0fb"
format_stamp: "Formatted at 2026-08-10 07:17:18 on dist-test-slave-10pc"
server_key: "34b16f73af2c4fa418a424bf9dc7d878"
server_key_iv: "deadbbf18d78d3dd1648826daa786753"
server_key_version: "encryptionkey@0"
I20260810 07:17:18.813912 30869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-1/wal/instance:
uuid: "77e8a9e544c54894841ccb51383ca0fb"
format_stamp: "Formatted at 2026-08-10 07:17:18 on dist-test-slave-10pc"
server_key: "34b16f73af2c4fa418a424bf9dc7d878"
server_key_iv: "deadbbf18d78d3dd1648826daa786753"
server_key_version: "encryptionkey@0"
I20260810 07:17:18.814582 30862 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.217.125" port: 39791 } attempt: 2
I20260810 07:17:18.814993 30865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:18.814855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:39791 (local address 127.0.0.1:52902)
0810 07:17:18.814923 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:18.814962 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.125:39791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 07:17:18.815436 30865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:18.815324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:39791 (local address 127.0.0.1:52914)
0810 07:17:18.815379 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:18.815406 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.125:39791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260810 07:17:18.815519 30869 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
W20260810 07:17:18.815568 30862 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.217.125:39791: Network error: Client connection negotiation failed: client connection to 127.15.217.125:39791: connect: Connection refused (error 111)
I20260810 07:17:18.816323 30886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:18.816550 30869 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:17:18.816650 30869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-1/wal
uuid: "77e8a9e544c54894841ccb51383ca0fb"
format_stamp: "Formatted at 2026-08-10 07:17:18 on dist-test-slave-10pc"
server_key: "34b16f73af2c4fa418a424bf9dc7d878"
server_key_iv: "deadbbf18d78d3dd1648826daa786753"
server_key_version: "encryptionkey@0"
I20260810 07:17:18.816728 30869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:18.831619 30869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:18.834219 30869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:18.834415 30869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:18.838737 30869 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.125:39791
I20260810 07:17:18.838783 30945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.125:39791 every 8 connection(s)
I20260810 07:17:18.839198 30869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-1/data/info.pb
I20260810 07:17:18.840142 30946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:17:18.841830 30946 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.217.126" port: 33243 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:17:18.847712 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 30869
Aug 10 07:17:18 dist-test-slave-10pc krb5kdc[30772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346238, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.125@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:17:18.847841 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-1/wal/instance
I20260810 07:17:18.848037 16229 external_mini_cluster.cc:1428] Setting key 1e9b45598506658e328e0e95b7edf252
I20260810 07:17:18.851722 30864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:18.842610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52366 (local address 127.15.217.126:33243)
0810 07:17:18.842683 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:18.842685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:18.842919 (+ 234us) server_negotiation.cc:407] Connection header received
0810 07:17:18.843368 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:18.843369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:18.843383 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:18.843439 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:18.844063 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:18.844631 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:18.845605 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:18.845815 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:18.845873 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:18.848433 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:18.848434 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:18.848434 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:18.848441 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:18.850020 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:18.850560 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:18.850561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:18.850561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:18.850615 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:18.850913 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:18.850913 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:18.850913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:18.851020 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:18.851072 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:18.851478 (+ 406us) server_negotiation.cc:299] Negotiation successful
0810 07:17:18.851580 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 07:17:18.851733 30950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:18.842600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:33243 (local address 127.0.0.1:52366)
0810 07:17:18.842794 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:18.842831 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:18.843284 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:18.843518 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:18.843521 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:18.843782 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:18.843983 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:18.843987 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:18.844711 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:18.844712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:18.845516 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:18.845518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:18.845895 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:18.845895 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:18.846006 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:18.846499 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:18.846507 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:18.848343 (+ 1836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:18.850109 (+ 1766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:18.850118 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:18.850121 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:18.850469 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:18.850685 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:18.850686 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:18.850687 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:18.850845 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:18.851095 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:18.851102 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:18.851298 (+ 196us) client_negotiation.cc:769] Sending connection context
0810 07:17:18.851427 (+ 129us) client_negotiation.cc:240] Negotiation successful
0810 07:17:18.851560 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":107,"threads_started":1}
I20260810 07:17:18.852480 30946 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.217.125" port: 39791 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:17:18 dist-test-slave-10pc krb5kdc[30772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346238, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.125@KRBTEST.COM for kudu/127.15.217.125@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.217.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.124@KRBTEST.COM" created.
I20260810 07:17:18.860605 30950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:18.852774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:39791 (local address 127.0.0.1:52928)
0810 07:17:18.853051 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:18.853064 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:18.853122 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:18.853291 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:18.853292 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:18.853464 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:18.853536 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:18.853538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:18.854374 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:18.854376 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:18.855272 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:18.855275 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:18.855918 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:18.855944 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:18.856055 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:18.856518 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:18.856523 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:18.857657 (+ 1134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:18.859521 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:18.859522 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:18.859522 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:18.859765 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:18.859982 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:18.859982 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:18.859983 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:18.860039 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:18.860300 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:18.860301 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:18.860339 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 07:17:18.860392 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 07:17:18.860537 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260}
I20260810 07:17:18.860773 30952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:18.852856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52928 (local address 127.15.217.125:39791)
0810 07:17:18.853061 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:18.853065 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:18.853137 (+ 72us) server_negotiation.cc:407] Connection header received
0810 07:17:18.853183 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:18.853187 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:18.853203 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:18.853275 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:18.853612 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:18.854284 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:18.855367 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:18.855829 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:18.855919 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:18.857777 (+ 1858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:18.857789 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:18.857789 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:18.857797 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:18.859441 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:18.859866 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:18.859868 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:18.859869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:18.859922 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:18.860087 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:18.860087 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:18.860087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:18.860227 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:18.860323 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:18.860431 (+ 108us) server_negotiation.cc:299] Negotiation successful
0810 07:17:18.860614 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":165,"thread_start_us":121,"threads_started":1}
I20260810 07:17:18.861605 30946 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42533 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:17:18.861955 30950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:18.861842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:42533 (local address 127.0.0.1:51410)
0810 07:17:18.861889 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:18.861935 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.124:42533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260810 07:17:18.862123 30891 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.217.124:42533: connect: Connection refused (error 111)
I20260810 07:17:18.862608 30950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:18.862511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:42533 (local address 127.0.0.1:51424)
0810 07:17:18.862562 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:18.862586 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.124:42533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 07:17:18.862730 30946 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.217.124:42533: Network error: Client connection negotiation failed: client connection to 127.15.217.124:42533: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.124.keytab.
Entry for principal kudu/127.15.217.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.124.keytab.
I20260810 07:17:18.875180 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.124:42533
--webserver_interface=127.15.217.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.124.keytab
--principal=kudu/127.15.217.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:45739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:33243,127.15.217.125:39791,127.15.217.124:42533
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:17:18.906728 30862 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.217.125" port: 39791 } attempt: 3
Aug 10 07:17:18 dist-test-slave-10pc krb5kdc[30772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346238, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for kudu/127.15.217.125@KRBTEST.COM
I20260810 07:17:18.911948 30946 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42533 } attempt: 1
I20260810 07:17:18.912428 30950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:18.912276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:42533 (local address 127.0.0.1:51440)
0810 07:17:18.912342 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:18.912381 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.124:42533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 07:17:18.913048 30950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:18.912931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:42533 (local address 127.0.0.1:51456)
0810 07:17:18.912987 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:18.913019 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.124:42533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260810 07:17:18.913149 30946 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.217.124:42533: Network error: Client connection negotiation failed: client connection to 127.15.217.124:42533: connect: Connection refused (error 111)
I20260810 07:17:18.914595 30952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:18.907250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52940 (local address 127.15.217.125:39791)
0810 07:17:18.907324 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:18.907326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:18.907340 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:17:18.907397 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:18.907398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:18.907413 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:18.907481 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:18.907849 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:18.908366 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:18.909113 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:18.909342 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:18.909424 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:18.911585 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:18.911586 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:18.911586 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:18.911595 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:18.913435 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:18.913791 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:18.913792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:18.913793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:18.913856 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:18.914062 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:18.914063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:18.914064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:18.914190 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:18.914269 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:18.914428 (+ 159us) server_negotiation.cc:299] Negotiation successful
0810 07:17:18.914513 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 07:17:18.914624 30865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:18.907163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:39791 (local address 127.0.0.1:52940)
0810 07:17:18.907236 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:18.907250 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:18.907332 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:18.907490 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:18.907491 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:18.907677 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:18.907754 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:18.907756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:18.908476 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:18.908478 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:18.909012 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:18.909014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:18.909422 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:18.909423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:18.909513 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:18.910009 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:18.910014 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:18.911489 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:18.913528 (+ 2039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:18.913530 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:18.913530 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:18.913709 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:18.913932 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:18.913933 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:18.913933 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:18.913995 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:18.914261 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:18.914262 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:18.914315 (+ 53us) client_negotiation.cc:769] Sending connection context
0810 07:17:18.914402 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 07:17:18.914513 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 07:17:18.915309 30862 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42533 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:17:18.915643 30865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:18.915556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:42533 (local address 127.0.0.1:51458)
0810 07:17:18.915589 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:18.915620 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.124:42533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260810 07:17:18.915930 30865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:18.915877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:42533 (local address 127.0.0.1:51468)
0810 07:17:18.915898 (+ 21us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:18.915917 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.124:42533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260810 07:17:18.916028 30862 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.217.124:42533: Network error: Client connection negotiation failed: client connection to 127.15.217.124:42533: connect: Connection refused (error 111)
I20260810 07:17:18.947189 30862 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42533 } attempt: 1
I20260810 07:17:18.947860 30865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:18.947670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:42533 (local address 127.0.0.1:51482)
0810 07:17:18.947765 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:18.947824 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.124:42533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260810 07:17:18.948382 30865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:18.948224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:42533 (local address 127.0.0.1:51496)
0810 07:17:18.948307 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:18.948354 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.124:42533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260810 07:17:18.948616 30862 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.217.124:42533: Network error: Client connection negotiation failed: client connection to 127.15.217.124:42533: connect: Connection refused (error 111)
W20260810 07:17:18.964553 30956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:18.964789 30956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:18.964812 30956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:18.966006 30956 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:17:18.966053 30956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:18.966068 30956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:18.966081 30956 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:17:18.966094 30956 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:17:18.967677 30956 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:45739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:33243,127.15.217.125:39791,127.15.217.124:42533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.124.keytab
--principal=kudu/127.15.217.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.124:42533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.217.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:17:18.967957 30956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:18.968217 30956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:18.973810 30965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:18.973831 30961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:18.974078 30963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:18.973979 30956 server_base.cc:1034] running on GCE node
I20260810 07:17:18.974370 30956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:18.974711 30956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:18.975909 30956 hybrid_clock.cc:648] HybridClock initialized: now 1786346238975863 us; error 35 us; skew 500 ppm
Aug 10 07:17:18 dist-test-slave-10pc krb5kdc[30772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346238, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:18.978642 30956 init.cc:377] Logged in from keytab as kudu/127.15.217.124@KRBTEST.COM (short username kudu)
I20260810 07:17:18.979149 30956 webserver.cc:460] Webserver started at http://127.15.217.124:41467/ using document root <none> and password file <none>
I20260810 07:17:18.979386 30956 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:18.979455 30956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:18.979619 30956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:17:18.980957 30956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-2/data/instance:
uuid: "8c0dfaffc21a479d8dd7f9065a9ed2a3"
format_stamp: "Formatted at 2026-08-10 07:17:18 on dist-test-slave-10pc"
server_key: "d5ffc41c590e99a4b961ee91c77ffe1e"
server_key_iv: "3af2039cd13d0b34df95add1008aa3e9"
server_key_version: "encryptionkey@0"
I20260810 07:17:18.981290 30946 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42533 } attempt: 2
I20260810 07:17:18.981427 30956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-2/wal/instance:
uuid: "8c0dfaffc21a479d8dd7f9065a9ed2a3"
format_stamp: "Formatted at 2026-08-10 07:17:18 on dist-test-slave-10pc"
server_key: "d5ffc41c590e99a4b961ee91c77ffe1e"
server_key_iv: "3af2039cd13d0b34df95add1008aa3e9"
server_key_version: "encryptionkey@0"
I20260810 07:17:18.981721 30950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:18.981580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:42533 (local address 127.0.0.1:51500)
0810 07:17:18.981634 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:18.981679 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.124:42533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260810 07:17:18.982107 30950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:18.982003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:42533 (local address 127.0.0.1:51516)
0810 07:17:18.982053 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:18.982081 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.124:42533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 07:17:18.982242 30946 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.217.124:42533: Network error: Client connection negotiation failed: client connection to 127.15.217.124:42533: connect: Connection refused (error 111)
I20260810 07:17:18.983136 30956 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:17:18.983917 30972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:18.984160 30956 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:17:18.984275 30956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-2/wal
uuid: "8c0dfaffc21a479d8dd7f9065a9ed2a3"
format_stamp: "Formatted at 2026-08-10 07:17:18 on dist-test-slave-10pc"
server_key: "d5ffc41c590e99a4b961ee91c77ffe1e"
server_key_iv: "3af2039cd13d0b34df95add1008aa3e9"
server_key_version: "encryptionkey@0"
I20260810 07:17:18.984377 30956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:19.002592 30956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:19.005048 30956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:19.005228 30956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:19.009558 30956 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.124:42533
I20260810 07:17:19.009615 31026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.124:42533 every 8 connection(s)
I20260810 07:17:19.010013 30956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-2/data/info.pb
I20260810 07:17:19.010912 31027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:17:19.011237 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 30956
I20260810 07:17:19.011370 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346081759552-16229-0/minicluster-data/master-2/wal/instance
I20260810 07:17:19.011569 16229 external_mini_cluster.cc:1428] Setting key ffd5ee367324b38e934bc4bbed55d434
I20260810 07:17:19.013149 31027 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.217.126" port: 33243 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:17:19 dist-test-slave-10pc krb5kdc[30772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346238, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.124@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
Aug 10 07:17:19 dist-test-slave-10pc krb5kdc[30772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.124@KRBTEST.COM
I20260810 07:17:19.023370 30864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:19.013946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52368 (local address 127.15.217.126:33243)
0810 07:17:19.014005 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:19.014007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:19.014242 (+ 235us) server_negotiation.cc:407] Connection header received
0810 07:17:19.014307 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:19.014308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:19.014320 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:19.014372 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:19.014838 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:19.015431 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:19.016480 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:19.016754 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:19.016822 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:19.019235 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:19.019236 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:19.019236 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:19.019243 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:19.021996 (+ 2753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:19.022438 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:19.022439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:19.022439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:19.022490 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:19.022743 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:19.022744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:19.022744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:19.022877 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:19.022955 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:19.023183 (+ 228us) server_negotiation.cc:299] Negotiation successful
0810 07:17:19.023272 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 07:17:19.023591 31031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:19.013914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:33243 (local address 127.0.0.1:52368)
0810 07:17:19.014131 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:19.014171 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:19.014245 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:19.014398 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:19.014400 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:19.014656 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:19.014751 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:19.014755 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:19.015525 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:19.015526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:19.016379 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:19.016382 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:19.016848 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:19.016849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:19.016930 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:19.017328 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:19.017339 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:19.019121 (+ 1782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:19.022077 (+ 2956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:19.022085 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:19.022088 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:19.022342 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:19.022565 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:19.022566 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:19.022567 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:19.022675 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:19.022966 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:19.022974 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:19.023046 (+ 72us) client_negotiation.cc:769] Sending connection context
0810 07:17:19.023146 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 07:17:19.023378 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":63,"thread_start_us":156,"threads_started":1}
I20260810 07:17:19.023762 30862 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42533 } attempt: 2
I20260810 07:17:19.023836 31030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:19.012206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51520 (local address 127.15.217.124:42533)
0810 07:17:19.012393 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:19.012403 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:19.012446 (+ 43us) server_negotiation.cc:407] Connection header received
0810 07:17:19.012967 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:19.012974 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:19.013025 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:19.013149 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:19.013934 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:19.015082 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:19.016102 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:19.016576 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:19.016689 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:19.019828 (+ 3139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:19.019837 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:19.019837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:19.019859 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:19.021799 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:19.022248 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:19.022251 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:19.022251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:19.022334 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:19.022560 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:19.022561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:19.022562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:19.022897 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:19.023031 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:19.023161 (+ 130us) server_negotiation.cc:299] Negotiation successful
0810 07:17:19.023294 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":104,"threads_started":1}
I20260810 07:17:19.024289 31027 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.217.125" port: 39791 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:17:19 dist-test-slave-10pc krb5kdc[30772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346238, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for kudu/127.15.217.124@KRBTEST.COM
Aug 10 07:17:19 dist-test-slave-10pc krb5kdc[30772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346238, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.124@KRBTEST.COM for kudu/127.15.217.125@KRBTEST.COM
Aug 10 07:17:19 dist-test-slave-10pc krb5kdc[30772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:17:19.031447 31030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:19.024063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51526 (local address 127.15.217.124:42533)
0810 07:17:19.024097 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:19.024099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:19.024415 (+ 316us) server_negotiation.cc:407] Connection header received
0810 07:17:19.024436 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:19.024437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:19.024470 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:19.024557 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:19.024865 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:19.025399 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:19.026061 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:19.026258 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:19.026333 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:19.028496 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:19.028497 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:19.028498 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:19.028505 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:19.030239 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:19.030645 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:19.030646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:19.030647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:19.030701 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:19.030914 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:19.030914 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:19.030915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:19.031029 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:19.031106 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:19.031269 (+ 163us) server_negotiation.cc:299] Negotiation successful
0810 07:17:19.031348 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260810 07:17:19.031450 30865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:19.024002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:42533 (local address 127.0.0.1:51526)
0810 07:17:19.024117 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:19.024127 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:19.024167 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:19.024574 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:19.024575 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:19.024746 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:19.024794 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:19.024796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:19.025482 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:19.025483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:19.025978 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:19.025979 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:19.026331 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:19.026332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:19.026390 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:19.026815 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:19.026821 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:19.028386 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:19.030345 (+ 1959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:19.030346 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:19.030346 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:19.030563 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:19.030779 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:19.030780 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:19.030780 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:19.030842 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:19.031099 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:19.031100 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:19.031154 (+ 54us) client_negotiation.cc:769] Sending connection context
0810 07:17:19.031242 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 07:17:19.031348 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":3}
I20260810 07:17:19.033536 30862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74ddf062a4dd43b7854141f53f9b0f0b: Bootstrap starting.
I20260810 07:17:19.034050 30864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:19.024769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52384 (local address 127.15.217.126:33243)
0810 07:17:19.024809 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:19.024811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:19.024864 (+ 53us) server_negotiation.cc:407] Connection header received
0810 07:17:19.024951 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:19.024952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:19.024960 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:19.025008 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:19.025628 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:19.026290 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:19.027232 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:19.027434 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:19.027519 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:19.030373 (+ 2854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:19.030374 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:19.030375 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:19.030381 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:19.032593 (+ 2212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:19.033006 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:19.033007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:19.033008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:19.033071 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:19.033316 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:19.033317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:19.033317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:19.033452 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:19.033636 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:19.033848 (+ 212us) server_negotiation.cc:299] Negotiation successful
0810 07:17:19.033937 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 07:17:19.034458 30952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:19.024790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52942 (local address 127.15.217.125:39791)
0810 07:17:19.024906 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:19.024909 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:19.024917 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:17:19.024961 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:19.024962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:19.024972 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:19.025022 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:19.025387 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:19.026136 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:19.027097 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:19.027417 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:19.027496 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:19.030340 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:19.030341 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:19.030342 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:19.030350 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:19.033152 (+ 2802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:19.033510 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:19.033511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:19.033511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:19.033578 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:19.033809 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:19.033810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:19.033811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:19.033950 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:19.034001 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:19.034250 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 07:17:19.034346 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260810 07:17:19.034456 30862 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74ddf062a4dd43b7854141f53f9b0f0b: Neither blocks nor log segments found. Creating new log.
I20260810 07:17:19.034775 31031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:19.024562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:39791 (local address 127.0.0.1:52942)
0810 07:17:19.024610 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:19.024619 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:19.024655 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:19.025074 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:19.025075 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:19.025224 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:19.025286 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:19.025288 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:19.026227 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:19.026228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:19.027006 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:19.027008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:19.027511 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:19.027512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:19.027592 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:19.027998 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:19.028002 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:19.030243 (+ 2241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:19.033221 (+ 2978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:19.033222 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:19.033222 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:19.033426 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:19.033667 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:19.033668 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:19.033668 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:19.033732 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:19.034059 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:19.034060 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:19.034109 (+ 49us) client_negotiation.cc:769] Sending connection context
0810 07:17:19.034551 (+ 442us) client_negotiation.cc:240] Negotiation successful
0810 07:17:19.034672 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260810 07:17:19.034817 30862 log.cc:824] T 00000000000000000000000000000000 P 74ddf062a4dd43b7854141f53f9b0f0b: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:19.035290 31027 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42533 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:17:19.035764 30862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74ddf062a4dd43b7854141f53f9b0f0b: No bootstrap required, opened a new log
I20260810 07:17:19.037840 30862 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74ddf062a4dd43b7854141f53f9b0f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ddf062a4dd43b7854141f53f9b0f0b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 33243 } } peers { permanent_uuid: "77e8a9e544c54894841ccb51383ca0fb" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 39791 } } peers { permanent_uuid: "8c0dfaffc21a479d8dd7f9065a9ed2a3" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42533 } }
I20260810 07:17:19.037990 30862 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74ddf062a4dd43b7854141f53f9b0f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:17:19.038028 30862 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74ddf062a4dd43b7854141f53f9b0f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74ddf062a4dd43b7854141f53f9b0f0b, State: Initialized, Role: FOLLOWER
I20260810 07:17:19.038163 30862 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74ddf062a4dd43b7854141f53f9b0f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ddf062a4dd43b7854141f53f9b0f0b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 33243 } } peers { permanent_uuid: "77e8a9e544c54894841ccb51383ca0fb" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 39791 } } peers { permanent_uuid: "8c0dfaffc21a479d8dd7f9065a9ed2a3" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42533 } }
I20260810 07:17:19.038516 30862 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74ddf062a4dd43b7854141f53f9b0f0b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:17:19.038456 31034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74ddf062a4dd43b7854141f53f9b0f0b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ddf062a4dd43b7854141f53f9b0f0b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 33243 } } peers { permanent_uuid: "77e8a9e544c54894841ccb51383ca0fb" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 39791 } } peers { permanent_uuid: "8c0dfaffc21a479d8dd7f9065a9ed2a3" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42533 } } }
I20260810 07:17:19.040333 31034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74ddf062a4dd43b7854141f53f9b0f0b [sys.catalog]: This master's current role is: FOLLOWER
W20260810 07:17:19.040668 31048 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 74ddf062a4dd43b7854141f53f9b0f0b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:17:19.040725 31048 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 10 07:17:19 dist-test-slave-10pc krb5kdc[30772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.125@KRBTEST.COM
Aug 10 07:17:19 dist-test-slave-10pc krb5kdc[30772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346238, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.124@KRBTEST.COM for kudu/127.15.217.124@KRBTEST.COM
I20260810 07:17:19.045078 30952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:19.035844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52950 (local address 127.15.217.125:39791)
0810 07:17:19.035888 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:19.035891 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:19.035899 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:17:19.035920 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:19.035921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:19.035930 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:19.035986 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:19.036423 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:19.037171 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:19.037952 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:19.038228 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:19.038306 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:19.041869 (+ 3563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:19.041870 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:19.041871 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:19.041889 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:19.043582 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:19.044015 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:19.044016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:19.044016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:19.044075 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:19.044288 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:19.044288 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:19.044289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:19.044417 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:19.044633 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:19.044867 (+ 234us) server_negotiation.cc:299] Negotiation successful
0810 07:17:19.044986 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 07:17:19.045218 31031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:19.035586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:42533 (local address 127.0.0.1:51534)
0810 07:17:19.035632 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:19.035642 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:19.035675 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:19.036681 (+ 1006us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:19.036682 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:19.036812 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:19.036838 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:19.036840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:19.037552 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:19.037553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:19.038276 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:19.038280 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:19.039061 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:19.039062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:19.039131 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:19.039575 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:19.039581 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:19.041690 (+ 2109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:19.044027 (+ 2337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:19.044028 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:19.044028 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:19.044258 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:19.044454 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:19.044454 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:19.044454 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:19.044552 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:19.044890 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:19.044891 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:19.044948 (+ 57us) client_negotiation.cc:769] Sending connection context
0810 07:17:19.045009 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 07:17:19.045109 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260810 07:17:19.045469 31030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:19.035617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51534 (local address 127.15.217.124:42533)
0810 07:17:19.036575 (+ 958us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:19.036577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:19.036583 (+ 6us) server_negotiation.cc:407] Connection header received
0810 07:17:19.036599 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:19.036600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:19.036608 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:19.036665 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:19.036917 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:19.037460 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:19.038370 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:19.038971 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:19.039050 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:19.041782 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:19.041783 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:19.041784 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:19.041791 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:19.043935 (+ 2144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:19.044340 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:19.044340 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:19.044341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:19.044393 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:19.044617 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:19.044617 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:19.044618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:19.044803 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:19.044869 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:19.045326 (+ 457us) server_negotiation.cc:299] Negotiation successful
0810 07:17:19.045395 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":933}
I20260810 07:17:19.041913 30787 master_runner.cc:186] Error getting master registration for 127.15.217.125:39791: OK, instance_id { permanent_uuid: "77e8a9e544c54894841ccb51383ca0fb" instance_seqno: 1786346238837556 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 07:17:19.046203 30787 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:17:19.046983 31027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c0dfaffc21a479d8dd7f9065a9ed2a3: Bootstrap starting.
I20260810 07:17:19.047875 31027 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c0dfaffc21a479d8dd7f9065a9ed2a3: Neither blocks nor log segments found. Creating new log.
I20260810 07:17:19.048245 31027 log.cc:824] T 00000000000000000000000000000000 P 8c0dfaffc21a479d8dd7f9065a9ed2a3: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:19.049256 31027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c0dfaffc21a479d8dd7f9065a9ed2a3: No bootstrap required, opened a new log
I20260810 07:17:19.050751 31027 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c0dfaffc21a479d8dd7f9065a9ed2a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ddf062a4dd43b7854141f53f9b0f0b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 33243 } } peers { permanent_uuid: "77e8a9e544c54894841ccb51383ca0fb" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 39791 } } peers { permanent_uuid: "8c0dfaffc21a479d8dd7f9065a9ed2a3" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42533 } }
I20260810 07:17:19.050868 31027 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c0dfaffc21a479d8dd7f9065a9ed2a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:17:19.050889 31027 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c0dfaffc21a479d8dd7f9065a9ed2a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c0dfaffc21a479d8dd7f9065a9ed2a3, State: Initialized, Role: FOLLOWER
I20260810 07:17:19.050987 31027 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c0dfaffc21a479d8dd7f9065a9ed2a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ddf062a4dd43b7854141f53f9b0f0b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 33243 } } peers { permanent_uuid: "77e8a9e544c54894841ccb51383ca0fb" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 39791 } } peers { permanent_uuid: "8c0dfaffc21a479d8dd7f9065a9ed2a3" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42533 } }
I20260810 07:17:19.051246 31051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c0dfaffc21a479d8dd7f9065a9ed2a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ddf062a4dd43b7854141f53f9b0f0b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 33243 } } peers { permanent_uuid: "77e8a9e544c54894841ccb51383ca0fb" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 39791 } } peers { permanent_uuid: "8c0dfaffc21a479d8dd7f9065a9ed2a3" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42533 } } }
I20260810 07:17:19.051306 31027 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c0dfaffc21a479d8dd7f9065a9ed2a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:17:19.051316 31051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c0dfaffc21a479d8dd7f9065a9ed2a3 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:17:19.053193 16229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260810 07:17:19.053608 31062 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8c0dfaffc21a479d8dd7f9065a9ed2a3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:17:19.053674 31062 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:17:19.056393 30956 master_runner.cc:186] Error getting master registration for 127.15.217.125:39791: OK, instance_id { permanent_uuid: "77e8a9e544c54894841ccb51383ca0fb" instance_seqno: 1786346238837556 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 07:17:19.059514 30956 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:17:19.065802 30864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:19.054874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52392 (local address 127.15.217.126:33243)
0810 07:17:19.054919 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:19.054921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:19.055516 (+ 595us) server_negotiation.cc:407] Connection header received
0810 07:17:19.055541 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:19.055542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:19.055555 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:19.055613 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:19.057045 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:19.057789 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:19.059011 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:19.059301 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:19.059345 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:19.061251 (+ 1906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:19.061253 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:19.061253 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:19.061261 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:19.063635 (+ 2374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:19.064828 (+ 1193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:19.064830 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:19.064830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:19.064902 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:19.065177 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:19.065178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:19.065178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:19.065315 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:19.065414 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:19.065614 (+ 200us) server_negotiation.cc:299] Negotiation successful
0810 07:17:19.065700 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 07:17:19.068593 30952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:19.055392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52952 (local address 127.15.217.125:39791)
0810 07:17:19.055494 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:19.055496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:19.056060 (+ 564us) server_negotiation.cc:407] Connection header received
0810 07:17:19.056091 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:19.056092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:19.056103 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:19.056163 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:19.061309 (+ 5146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:19.062361 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:19.063697 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:19.064000 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:19.064041 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:19.065596 (+ 1555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:19.065598 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:19.065598 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:19.065607 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:19.067287 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:19.067687 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:19.067688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:19.067689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:19.067745 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:19.067937 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:19.067938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:19.067938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:19.068064 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:19.068166 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:19.068382 (+ 216us) server_negotiation.cc:299] Negotiation successful
0810 07:17:19.068491 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 07:17:19.071585 31030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:19.055708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51550 (local address 127.15.217.124:42533)
0810 07:17:19.055898 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:19.055900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:19.055998 (+ 98us) server_negotiation.cc:407] Connection header received
0810 07:17:19.060334 (+ 4336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:19.060336 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:19.060351 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:19.060384 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:19.061586 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:19.062308 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:19.066751 (+ 4443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:19.067057 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:19.067105 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:19.068575 (+ 1470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:19.068577 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:19.068577 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:19.068585 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:19.070294 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:19.070714 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:19.070715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:19.070716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:19.070777 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:19.071001 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:19.071002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:19.071003 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:19.071142 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:19.071234 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:19.071400 (+ 166us) server_negotiation.cc:299] Negotiation successful
0810 07:17:19.071484 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170}
I20260810 07:17:19.073402 30946 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42533 } attempt: 3
Aug 10 07:17:19 dist-test-slave-10pc krb5kdc[30772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346238, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.125@KRBTEST.COM for kudu/127.15.217.124@KRBTEST.COM
I20260810 07:17:19.081521 30950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:19.073664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:42533 (local address 127.0.0.1:51566)
0810 07:17:19.073707 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:19.073720 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:19.073764 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:19.074082 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:19.074083 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:19.074250 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:19.074296 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:19.074297 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:19.075213 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:19.075214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:19.076033 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:19.076036 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:19.076552 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:19.076553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:19.076618 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:19.077076 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:19.077081 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:19.078600 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:19.080399 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:19.080400 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:19.080400 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:19.080644 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:19.080862 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:19.080862 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:19.080863 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:19.080919 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:19.081204 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:19.081205 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:19.081252 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 07:17:19.081314 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 07:17:19.081410 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260810 07:17:19.081871 31030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:19.073769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51566 (local address 127.15.217.124:42533)
0810 07:17:19.073938 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:19.073940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:19.073947 (+ 7us) server_negotiation.cc:407] Connection header received
0810 07:17:19.073975 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:19.073975 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:19.073985 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:19.074036 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:19.074378 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:19.075114 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:19.076132 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:19.076442 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:19.076542 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:19.078690 (+ 2148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:19.078692 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:19.078693 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:19.078700 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:19.080311 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:19.080735 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:19.080736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:19.080737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:19.080796 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:19.080980 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:19.080980 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:19.080981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:19.081105 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:19.081175 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:19.081704 (+ 529us) server_negotiation.cc:299] Negotiation successful
0810 07:17:19.081781 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150}
I20260810 07:17:19.083452 30946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77e8a9e544c54894841ccb51383ca0fb: Bootstrap starting.
I20260810 07:17:19.084270 30946 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77e8a9e544c54894841ccb51383ca0fb: Neither blocks nor log segments found. Creating new log.
I20260810 07:17:19.084648 30946 log.cc:824] T 00000000000000000000000000000000 P 77e8a9e544c54894841ccb51383ca0fb: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:19.085667 30946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77e8a9e544c54894841ccb51383ca0fb: No bootstrap required, opened a new log
I20260810 07:17:19.088487 30946 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77e8a9e544c54894841ccb51383ca0fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ddf062a4dd43b7854141f53f9b0f0b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 33243 } } peers { permanent_uuid: "77e8a9e544c54894841ccb51383ca0fb" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 39791 } } peers { permanent_uuid: "8c0dfaffc21a479d8dd7f9065a9ed2a3" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42533 } }
I20260810 07:17:19.088637 30946 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77e8a9e544c54894841ccb51383ca0fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:17:19.088671 30946 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77e8a9e544c54894841ccb51383ca0fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77e8a9e544c54894841ccb51383ca0fb, State: Initialized, Role: FOLLOWER
I20260810 07:17:19.088796 30946 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77e8a9e544c54894841ccb51383ca0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ddf062a4dd43b7854141f53f9b0f0b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 33243 } } peers { permanent_uuid: "77e8a9e544c54894841ccb51383ca0fb" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 39791 } } peers { permanent_uuid: "8c0dfaffc21a479d8dd7f9065a9ed2a3" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42533 } }
I20260810 07:17:19.089154 30946 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77e8a9e544c54894841ccb51383ca0fb [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:17:19.089118 31074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77e8a9e544c54894841ccb51383ca0fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ddf062a4dd43b7854141f53f9b0f0b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 33243 } } peers { permanent_uuid: "77e8a9e544c54894841ccb51383ca0fb" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 39791 } } peers { permanent_uuid: "8c0dfaffc21a479d8dd7f9065a9ed2a3" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42533 } } }
I20260810 07:17:19.089318 31074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77e8a9e544c54894841ccb51383ca0fb [sys.catalog]: This master's current role is: FOLLOWER
W20260810 07:17:19.091087 31086 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 77e8a9e544c54894841ccb51383ca0fb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:17:19.091205 31086 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:17:19.093775 30869 master_runner.cc:263] No leader master found from master 77e8a9e544c54894841ccb51383ca0fb
I20260810 07:17:19.093839 30869 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:17:19.099982 31034 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74ddf062a4dd43b7854141f53f9b0f0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:17:19.100102 31034 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74ddf062a4dd43b7854141f53f9b0f0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ddf062a4dd43b7854141f53f9b0f0b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 33243 } } peers { permanent_uuid: "77e8a9e544c54894841ccb51383ca0fb" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 39791 } } peers { permanent_uuid: "8c0dfaffc21a479d8dd7f9065a9ed2a3" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42533 } }
I20260810 07:17:19.100346 31034 leader_election.cc:288] T 00000000000000000000000000000000 P 74ddf062a4dd43b7854141f53f9b0f0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 77e8a9e544c54894841ccb51383ca0fb (127.15.217.125:39791), 8c0dfaffc21a479d8dd7f9065a9ed2a3 (127.15.217.124:42533)
I20260810 07:17:19.100821 30918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "74ddf062a4dd43b7854141f53f9b0f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77e8a9e544c54894841ccb51383ca0fb" is_pre_election: true
I20260810 07:17:19.100991 30918 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 77e8a9e544c54894841ccb51383ca0fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74ddf062a4dd43b7854141f53f9b0f0b in term 0.
I20260810 07:17:19.101184 31000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "74ddf062a4dd43b7854141f53f9b0f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c0dfaffc21a479d8dd7f9065a9ed2a3" is_pre_election: true
I20260810 07:17:19.101342 31000 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8c0dfaffc21a479d8dd7f9065a9ed2a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74ddf062a4dd43b7854141f53f9b0f0b in term 0.
I20260810 07:17:19.101574 30809 leader_election.cc:302] T 00000000000000000000000000000000 P 74ddf062a4dd43b7854141f53f9b0f0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74ddf062a4dd43b7854141f53f9b0f0b, 77e8a9e544c54894841ccb51383ca0fb; no voters:
I20260810 07:17:19.102520 31034 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74ddf062a4dd43b7854141f53f9b0f0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:17:19.102598 31034 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74ddf062a4dd43b7854141f53f9b0f0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:17:19.102624 31034 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74ddf062a4dd43b7854141f53f9b0f0b [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:17:19.103926 31034 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74ddf062a4dd43b7854141f53f9b0f0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ddf062a4dd43b7854141f53f9b0f0b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 33243 } } peers { permanent_uuid: "77e8a9e544c54894841ccb51383ca0fb" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 39791 } } peers { permanent_uuid: "8c0dfaffc21a479d8dd7f9065a9ed2a3" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42533 } }
I20260810 07:17:19.104064 31034 leader_election.cc:288] T 00000000000000000000000000000000 P 74ddf062a4dd43b7854141f53f9b0f0b [CANDIDATE]: Term 1 election: Requested vote from peers 77e8a9e544c54894841ccb51383ca0fb (127.15.217.125:39791), 8c0dfaffc21a479d8dd7f9065a9ed2a3 (127.15.217.124:42533)
I20260810 07:17:19.104511 31000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "74ddf062a4dd43b7854141f53f9b0f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c0dfaffc21a479d8dd7f9065a9ed2a3"
I20260810 07:17:19.104596 31000 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c0dfaffc21a479d8dd7f9065a9ed2a3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:17:19.105535 31000 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8c0dfaffc21a479d8dd7f9065a9ed2a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74ddf062a4dd43b7854141f53f9b0f0b in term 1.
I20260810 07:17:19.105701 30918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "74ddf062a4dd43b7854141f53f9b0f0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77e8a9e544c54894841ccb51383ca0fb"
I20260810 07:17:19.105782 30918 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77e8a9e544c54894841ccb51383ca0fb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:17:19.106629 30918 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 77e8a9e544c54894841ccb51383ca0fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74ddf062a4dd43b7854141f53f9b0f0b in term 1.
I20260810 07:17:19.106902 30809 leader_election.cc:302] T 00000000000000000000000000000000 P 74ddf062a4dd43b7854141f53f9b0f0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74ddf062a4dd43b7854141f53f9b0f0b, 77e8a9e544c54894841ccb51383ca0fb; no voters:
I20260810 07:17:19.107017 31034 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74ddf062a4dd43b7854141f53f9b0f0b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:17:19.107196 31034 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74ddf062a4dd43b7854141f53f9b0f0b [term 1 LEADER]: Becoming Leader. State: Replica: 74ddf062a4dd43b7854141f53f9b0f0b, State: Running, Role: LEADER
I20260810 07:17:19.107288 31034 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74ddf062a4dd43b7854141f53f9b0f0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ddf062a4dd43b7854141f53f9b0f0b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 33243 } } peers { permanent_uuid: "77e8a9e544c54894841ccb51383ca0fb" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 39791 } } peers { permanent_uuid: "8c0dfaffc21a479d8dd7f9065a9ed2a3" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42533 } }
I20260810 07:17:19.107668 31034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74ddf062a4dd43b7854141f53f9b0f0b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74ddf062a4dd43b7854141f53f9b0f0b. Latest consensus state: current_term: 1 leader_uuid: "74ddf062a4dd43b7854141f53f9b0f0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ddf062a4dd43b7854141f53f9b0f0b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 33243 } } peers { permanent_uuid: "77e8a9e544c54894841ccb51383ca0fb" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 39791 } } peers { permanent_uuid: "8c0dfaffc21a479d8dd7f9065a9ed2a3" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42533 } } }
I20260810 07:17:19.107745 31034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74ddf062a4dd43b7854141f53f9b0f0b [sys.catalog]: This master's current role is: LEADER
I20260810 07:17:19.107950 31090 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:17:19.108163 31090 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:17:19.110121 31000 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8c0dfaffc21a479d8dd7f9065a9ed2a3 [term 1 FOLLOWER]: Refusing update from remote peer 74ddf062a4dd43b7854141f53f9b0f0b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:17:19.110407 31034 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 74ddf062a4dd43b7854141f53f9b0f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c0dfaffc21a479d8dd7f9065a9ed2a3" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:17:19.111618 31051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c0dfaffc21a479d8dd7f9065a9ed2a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74ddf062a4dd43b7854141f53f9b0f0b. Latest consensus state: current_term: 1 leader_uuid: "74ddf062a4dd43b7854141f53f9b0f0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ddf062a4dd43b7854141f53f9b0f0b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 33243 } } peers { permanent_uuid: "77e8a9e544c54894841ccb51383ca0fb" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 39791 } } peers { permanent_uuid: "8c0dfaffc21a479d8dd7f9065a9ed2a3" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42533 } } }
I20260810 07:17:19.111707 31051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c0dfaffc21a479d8dd7f9065a9ed2a3 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:17:19.112061 30918 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 77e8a9e544c54894841ccb51383ca0fb [term 1 FOLLOWER]: Refusing update from remote peer 74ddf062a4dd43b7854141f53f9b0f0b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:17:19.112317 31034 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 74ddf062a4dd43b7854141f53f9b0f0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "77e8a9e544c54894841ccb51383ca0fb" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 39791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:17:19.113499 31074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77e8a9e544c54894841ccb51383ca0fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74ddf062a4dd43b7854141f53f9b0f0b. Latest consensus state: current_term: 1 leader_uuid: "74ddf062a4dd43b7854141f53f9b0f0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ddf062a4dd43b7854141f53f9b0f0b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 33243 } } peers { permanent_uuid: "77e8a9e544c54894841ccb51383ca0fb" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 39791 } } peers { permanent_uuid: "8c0dfaffc21a479d8dd7f9065a9ed2a3" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42533 } } }
I20260810 07:17:19.113592 31074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77e8a9e544c54894841ccb51383ca0fb [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:17:19.115764 31091 mvcc.cc:205] Tried to move back new op lower bound from 7316874195392811008 to 7316874195384012800. Current Snapshot: MvccSnapshot[applied={T|T < 7316874195392811008}]
I20260810 07:17:19.116065 31034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74ddf062a4dd43b7854141f53f9b0f0b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "74ddf062a4dd43b7854141f53f9b0f0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ddf062a4dd43b7854141f53f9b0f0b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 33243 } } peers { permanent_uuid: "77e8a9e544c54894841ccb51383ca0fb" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 39791 } } peers { permanent_uuid: "8c0dfaffc21a479d8dd7f9065a9ed2a3" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42533 } } }
I20260810 07:17:19.116140 31034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74ddf062a4dd43b7854141f53f9b0f0b [sys.catalog]: This master's current role is: LEADER
I20260810 07:17:19.116567 31074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77e8a9e544c54894841ccb51383ca0fb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "74ddf062a4dd43b7854141f53f9b0f0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ddf062a4dd43b7854141f53f9b0f0b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 33243 } } peers { permanent_uuid: "77e8a9e544c54894841ccb51383ca0fb" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 39791 } } peers { permanent_uuid: "8c0dfaffc21a479d8dd7f9065a9ed2a3" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42533 } } }
I20260810 07:17:19.116650 31074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77e8a9e544c54894841ccb51383ca0fb [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:17:19.116932 31094 mvcc.cc:205] Tried to move back new op lower bound from 7316874195392811008 to 7316874195384012800. Current Snapshot: MvccSnapshot[applied={T|T < 7316874195392811008}]
I20260810 07:17:19.117715 31034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74ddf062a4dd43b7854141f53f9b0f0b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "74ddf062a4dd43b7854141f53f9b0f0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ddf062a4dd43b7854141f53f9b0f0b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 33243 } } peers { permanent_uuid: "77e8a9e544c54894841ccb51383ca0fb" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 39791 } } peers { permanent_uuid: "8c0dfaffc21a479d8dd7f9065a9ed2a3" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42533 } } }
I20260810 07:17:19.117794 31034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74ddf062a4dd43b7854141f53f9b0f0b [sys.catalog]: This master's current role is: LEADER
I20260810 07:17:19.118135 31051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c0dfaffc21a479d8dd7f9065a9ed2a3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "74ddf062a4dd43b7854141f53f9b0f0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74ddf062a4dd43b7854141f53f9b0f0b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 33243 } } peers { permanent_uuid: "77e8a9e544c54894841ccb51383ca0fb" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 39791 } } peers { permanent_uuid: "8c0dfaffc21a479d8dd7f9065a9ed2a3" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42533 } } }
I20260810 07:17:19.118221 31051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c0dfaffc21a479d8dd7f9065a9ed2a3 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:17:19.118852 31092 mvcc.cc:205] Tried to move back new op lower bound from 7316874195392811008 to 7316874195384012800. Current Snapshot: MvccSnapshot[applied={T|T < 7316874195392811008}]
I20260810 07:17:19.119695 31090 catalog_manager.cc:1347] Generated new cluster ID: 60358353e37f437f8f848dbf53243a3f
I20260810 07:17:19.119748 31090 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:17:19.139009 31090 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:17:19.139771 31090 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:17:19.150044 31090 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74ddf062a4dd43b7854141f53f9b0f0b: Generated new TSK 0
I20260810 07:17:19.150262 31090 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:17:20.054160 31062 catalog_manager.cc:1259] Loaded cluster ID: 60358353e37f437f8f848dbf53243a3f
I20260810 07:17:20.054242 31062 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8c0dfaffc21a479d8dd7f9065a9ed2a3: loading cluster ID for follower catalog manager: success
I20260810 07:17:20.055164 31062 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8c0dfaffc21a479d8dd7f9065a9ed2a3: acquiring CA information for follower catalog manager: success
I20260810 07:17:20.055507 31062 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8c0dfaffc21a479d8dd7f9065a9ed2a3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:17:20.091706 31086 catalog_manager.cc:1259] Loaded cluster ID: 60358353e37f437f8f848dbf53243a3f
I20260810 07:17:20.091791 31086 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 77e8a9e544c54894841ccb51383ca0fb: loading cluster ID for follower catalog manager: success
I20260810 07:17:20.092660 31086 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 77e8a9e544c54894841ccb51383ca0fb: acquiring CA information for follower catalog manager: success
I20260810 07:17:20.092998 31086 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 77e8a9e544c54894841ccb51383ca0fb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:17:20.204799 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 30787
I20260810 07:17:20.212872 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 30869
I20260810 07:17:20.219367 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 30956
2026-08-10T07:17:20Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2954 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:17:20 dist-test-slave-10pc krb5kdc[31110](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:17:20 dist-test-slave-10pc krb5kdc[31110](info): set up 2 sockets
Aug 10 07:17:20 dist-test-slave-10pc krb5kdc[31110](info): commencing operation
krb5kdc: starting...
W20260810 07:17:21.315116 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:17:21 dist-test-slave-10pc krb5kdc[31110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346241, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:17:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:17:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
I20260810 07:17:21.445693 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:33893
--webserver_interface=127.15.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:38463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:33893
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:17:21.532058 31123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:21.532279 31123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:21.532301 31123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:21.533452 31123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:17:21.533488 31123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:21.533502 31123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:21.533515 31123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:17:21.533527 31123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:17:21.534938 31123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:38463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:33893
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:33893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:17:21.535140 31123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:21.535329 31123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:21.540436 31129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:21.540493 31131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:21.540472 31128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:21.540624 31123 server_base.cc:1034] running on GCE node
I20260810 07:17:21.541078 31123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:21.541467 31123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:21.542650 31123 hybrid_clock.cc:648] HybridClock initialized: now 1786346241542624 us; error 36 us; skew 500 ppm
Aug 10 07:17:21 dist-test-slave-10pc krb5kdc[31110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346241, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:21.545320 31123 init.cc:377] Logged in from keytab as kudu/127.15.217.126@KRBTEST.COM (short username kudu)
I20260810 07:17:21.545792 31123 webserver.cc:460] Webserver started at http://127.15.217.126:43265/ using document root <none> and password file <none>
I20260810 07:17:21.546034 31123 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:21.546106 31123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:21.546268 31123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:17:21.547372 31123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "27892e8c00be41f9b159f9649a1c3753"
format_stamp: "Formatted at 2026-08-10 07:17:21 on dist-test-slave-10pc"
server_key: "bb38d655522ce63fded643753c0659e5"
server_key_iv: "c2b3e122afc793e1050c42d0d5266d87"
server_key_version: "encryptionkey@0"
I20260810 07:17:21.547771 31123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "27892e8c00be41f9b159f9649a1c3753"
format_stamp: "Formatted at 2026-08-10 07:17:21 on dist-test-slave-10pc"
server_key: "bb38d655522ce63fded643753c0659e5"
server_key_iv: "c2b3e122afc793e1050c42d0d5266d87"
server_key_version: "encryptionkey@0"
I20260810 07:17:21.549247 31123 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:17:21.550000 31138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:21.550185 31123 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 07:17:21.550297 31123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "27892e8c00be41f9b159f9649a1c3753"
format_stamp: "Formatted at 2026-08-10 07:17:21 on dist-test-slave-10pc"
server_key: "bb38d655522ce63fded643753c0659e5"
server_key_iv: "c2b3e122afc793e1050c42d0d5266d87"
server_key_version: "encryptionkey@0"
I20260810 07:17:21.550372 31123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:21.574590 31123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:21.576915 31123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:21.577102 31123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:21.581425 31123 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:33893
I20260810 07:17:21.581482 31197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:33893 every 8 connection(s)
I20260810 07:17:21.581952 31123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:17:21.582959 31198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:17:21.585443 31198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27892e8c00be41f9b159f9649a1c3753: Bootstrap starting.
I20260810 07:17:21.586088 31198 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27892e8c00be41f9b159f9649a1c3753: Neither blocks nor log segments found. Creating new log.
I20260810 07:17:21.586402 31198 log.cc:824] T 00000000000000000000000000000000 P 27892e8c00be41f9b159f9649a1c3753: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:21.587179 31198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27892e8c00be41f9b159f9649a1c3753: No bootstrap required, opened a new log
I20260810 07:17:21.588845 31198 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27892e8c00be41f9b159f9649a1c3753 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27892e8c00be41f9b159f9649a1c3753" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 33893 } }
I20260810 07:17:21.588996 31198 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27892e8c00be41f9b159f9649a1c3753 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:17:21.589047 31198 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27892e8c00be41f9b159f9649a1c3753 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27892e8c00be41f9b159f9649a1c3753, State: Initialized, Role: FOLLOWER
I20260810 07:17:21.589198 31198 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27892e8c00be41f9b159f9649a1c3753 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27892e8c00be41f9b159f9649a1c3753" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 33893 } }
I20260810 07:17:21.589263 31198 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27892e8c00be41f9b159f9649a1c3753 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:17:21.589284 31198 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27892e8c00be41f9b159f9649a1c3753 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:17:21.589357 31198 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27892e8c00be41f9b159f9649a1c3753 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:17:21.589888 31198 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27892e8c00be41f9b159f9649a1c3753 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27892e8c00be41f9b159f9649a1c3753" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 33893 } }
I20260810 07:17:21.590018 31198 leader_election.cc:302] T 00000000000000000000000000000000 P 27892e8c00be41f9b159f9649a1c3753 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27892e8c00be41f9b159f9649a1c3753; no voters:
I20260810 07:17:21.590224 31198 leader_election.cc:288] T 00000000000000000000000000000000 P 27892e8c00be41f9b159f9649a1c3753 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:17:21.590260 31201 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27892e8c00be41f9b159f9649a1c3753 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:17:21.590425 31201 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27892e8c00be41f9b159f9649a1c3753 [term 1 LEADER]: Becoming Leader. State: Replica: 27892e8c00be41f9b159f9649a1c3753, State: Running, Role: LEADER
I20260810 07:17:21.590502 31198 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27892e8c00be41f9b159f9649a1c3753 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:17:21.590520 31201 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27892e8c00be41f9b159f9649a1c3753 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27892e8c00be41f9b159f9649a1c3753" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 33893 } }
I20260810 07:17:21.590893 31202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27892e8c00be41f9b159f9649a1c3753 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27892e8c00be41f9b159f9649a1c3753" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27892e8c00be41f9b159f9649a1c3753" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 33893 } } }
I20260810 07:17:21.590983 31202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27892e8c00be41f9b159f9649a1c3753 [sys.catalog]: This master's current role is: LEADER
I20260810 07:17:21.591145 31203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27892e8c00be41f9b159f9649a1c3753 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27892e8c00be41f9b159f9649a1c3753. Latest consensus state: current_term: 1 leader_uuid: "27892e8c00be41f9b159f9649a1c3753" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27892e8c00be41f9b159f9649a1c3753" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 33893 } } }
I20260810 07:17:21.591250 31208 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:17:21.591270 31203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27892e8c00be41f9b159f9649a1c3753 [sys.catalog]: This master's current role is: LEADER
I20260810 07:17:21.591856 31208 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:17:21.592018 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 31123
I20260810 07:17:21.592121 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:17:21.592348 16229 external_mini_cluster.cc:1428] Setting key 9112fc7f7806cc15f4fc695f162c73cf
I20260810 07:17:21.593811 31208 catalog_manager.cc:1347] Generated new cluster ID: 377fbb12ff164011b29b54f9049fc35f
I20260810 07:17:21.593865 31208 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:17:21.598842 31208 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:17:21.599361 31208 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:17:21.604858 31208 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27892e8c00be41f9b159f9649a1c3753: Generated new TSK 0
I20260810 07:17:21.605113 31208 catalog_manager.cc:1514] Initializing in-progress tserver states...
Aug 10 07:17:21 dist-test-slave-10pc krb5kdc[31110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346241, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:17:21.610260 31224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:21.592803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58144 (local address 127.15.217.126:33893)
0810 07:17:21.593009 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:21.593019 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:21.593301 (+ 282us) server_negotiation.cc:407] Connection header received
0810 07:17:21.593444 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:21.593453 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:21.593511 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:21.593665 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:21.594634 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:21.596031 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:21.597194 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:21.597792 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:21.597898 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:21.606751 (+ 8853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:21.606756 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:21.606756 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:21.606769 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:21.608641 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:21.609024 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:21.609027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:21.609027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:21.609128 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:21.609349 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:21.609350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:21.609350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:21.609591 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:21.609759 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:21.609893 (+ 134us) server_negotiation.cc:299] Negotiation successful
0810 07:17:21.610070 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":96,"threads_started":1}
I20260810 07:17:21.611402 16229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:17:21.614682 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 31123
2026-08-10T07:17:21Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1396 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:17:21 dist-test-slave-10pc krb5kdc[31230](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:17:21 dist-test-slave-10pc krb5kdc[31230](info): set up 2 sockets
Aug 10 07:17:21 dist-test-slave-10pc krb5kdc[31230](info): commencing operation
krb5kdc: starting...
W20260810 07:17:22.717135 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:17:22 dist-test-slave-10pc krb5kdc[31230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:17:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:17:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
I20260810 07:17:22.847900 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:43519
--webserver_interface=127.15.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:38625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:43519
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:17:22.934175 31245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:22.934386 31245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:22.934408 31245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:22.935509 31245 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:17:22.935544 31245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:22.935559 31245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:22.935571 31245 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:17:22.935583 31245 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:17:22.937002 31245 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:38625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:43519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:43519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:17:22.937193 31245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:22.937377 31245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:22.942667 31250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:22.942709 31254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:22.942667 31252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:22.942927 31245 server_base.cc:1034] running on GCE node
I20260810 07:17:22.943176 31245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:22.943486 31245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:22.944648 31245 hybrid_clock.cc:648] HybridClock initialized: now 1786346242944633 us; error 49 us; skew 500 ppm
Aug 10 07:17:22 dist-test-slave-10pc krb5kdc[31230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346242, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:22.947295 31245 init.cc:377] Logged in from keytab as kudu/127.15.217.126@KRBTEST.COM (short username kudu)
I20260810 07:17:22.947800 31245 webserver.cc:460] Webserver started at http://127.15.217.126:43441/ using document root <none> and password file <none>
I20260810 07:17:22.948066 31245 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:22.948143 31245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:22.948297 31245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:17:22.949488 31245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "b5b017740ffd4355814fc558cbe98de3"
format_stamp: "Formatted at 2026-08-10 07:17:22 on dist-test-slave-10pc"
server_key: "9cf669d801a5ea9d14511819eaf4ab2b"
server_key_iv: "959cdcfc8ac39d4c29d7d9f1a271b1c8"
server_key_version: "encryptionkey@0"
I20260810 07:17:22.949882 31245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "b5b017740ffd4355814fc558cbe98de3"
format_stamp: "Formatted at 2026-08-10 07:17:22 on dist-test-slave-10pc"
server_key: "9cf669d801a5ea9d14511819eaf4ab2b"
server_key_iv: "959cdcfc8ac39d4c29d7d9f1a271b1c8"
server_key_version: "encryptionkey@0"
I20260810 07:17:22.951252 31245 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:17:22.952037 31263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:22.952215 31245 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 07:17:22.952322 31245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "b5b017740ffd4355814fc558cbe98de3"
format_stamp: "Formatted at 2026-08-10 07:17:22 on dist-test-slave-10pc"
server_key: "9cf669d801a5ea9d14511819eaf4ab2b"
server_key_iv: "959cdcfc8ac39d4c29d7d9f1a271b1c8"
server_key_version: "encryptionkey@0"
I20260810 07:17:22.952406 31245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:22.961637 31245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:22.963464 31245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:22.963636 31245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:22.968070 31245 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:43519
I20260810 07:17:22.968108 31320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:43519 every 8 connection(s)
I20260810 07:17:22.968523 31245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:17:22.969533 31321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:17:22.971864 31321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3: Bootstrap starting.
I20260810 07:17:22.972528 31321 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3: Neither blocks nor log segments found. Creating new log.
I20260810 07:17:22.972833 31321 log.cc:824] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:22.973538 31321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3: No bootstrap required, opened a new log
I20260810 07:17:22.974213 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 31245
I20260810 07:17:22.974305 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:17:22.974521 16229 external_mini_cluster.cc:1428] Setting key b6dc43f22b8fc0b73e7b3233c0de8101
I20260810 07:17:22.975723 31321 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b017740ffd4355814fc558cbe98de3" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 43519 } }
I20260810 07:17:22.975898 31321 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:17:22.975946 31321 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5b017740ffd4355814fc558cbe98de3, State: Initialized, Role: FOLLOWER
I20260810 07:17:22.976071 31321 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b017740ffd4355814fc558cbe98de3" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 43519 } }
I20260810 07:17:22.976176 31321 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:17:22.976223 31321 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:17:22.976272 31321 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:17:22.977219 31321 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b017740ffd4355814fc558cbe98de3" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 43519 } }
I20260810 07:17:22.977344 31321 leader_election.cc:302] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5b017740ffd4355814fc558cbe98de3; no voters:
I20260810 07:17:22.977522 31321 leader_election.cc:288] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:17:22.977567 31326 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:17:22.977733 31326 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [term 1 LEADER]: Becoming Leader. State: Replica: b5b017740ffd4355814fc558cbe98de3, State: Running, Role: LEADER
I20260810 07:17:22.977794 31321 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:17:22.977821 31326 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b017740ffd4355814fc558cbe98de3" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 43519 } }
I20260810 07:17:22.978120 31326 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5b017740ffd4355814fc558cbe98de3. Latest consensus state: current_term: 1 leader_uuid: "b5b017740ffd4355814fc558cbe98de3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b017740ffd4355814fc558cbe98de3" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 43519 } } }
I20260810 07:17:22.978129 31329 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5b017740ffd4355814fc558cbe98de3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b017740ffd4355814fc558cbe98de3" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 43519 } } }
I20260810 07:17:22.978214 31326 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [sys.catalog]: This master's current role is: LEADER
I20260810 07:17:22.978237 31329 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [sys.catalog]: This master's current role is: LEADER
W20260810 07:17:22.979733 31346 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:17:22.979790 31346 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:17:22.979846 31333 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:17:22.980047 31333 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:17:22.981669 31333 catalog_manager.cc:1347] Generated new cluster ID: 187ad450aac64b6492ee6c7a2add93c1
I20260810 07:17:22.981724 31333 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 07:17:22 dist-test-slave-10pc krb5kdc[31230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:17:22.986884 31325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:22.975200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39882 (local address 127.15.217.126:43519)
0810 07:17:22.975389 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:22.975397 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:22.975423 (+ 26us) server_negotiation.cc:407] Connection header received
0810 07:17:22.975727 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:22.975736 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:22.975799 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:22.975976 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:22.976818 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:22.978133 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:22.979296 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:22.979907 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:22.980044 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:22.982976 (+ 2932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:22.982980 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:22.982980 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:22.982993 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:22.985141 (+ 2148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:22.985583 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:22.985586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:22.985587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:22.985716 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:22.985936 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:22.985936 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:22.985937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:22.986247 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:22.986406 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:22.986528 (+ 122us) server_negotiation.cc:299] Negotiation successful
0810 07:17:22.986689 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":85,"threads_started":1}
I20260810 07:17:22.992664 31333 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:17:22.993331 31333 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:17:23.002470 31333 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3: Generated new TSK 0
I20260810 07:17:23.002676 31333 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab.
Entry for principal kudu/127.15.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab.
I20260810 07:17:23.038987 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.65:0
--local_ip_for_outbound_sockets=127.15.217.65
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:43519
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:38625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:17:23.124599 31351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:23.124805 31351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:23.124831 31351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:23.125986 31351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:23.126027 31351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:23.126053 31351 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.65
I20260810 07:17:23.127471 31351 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:38625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:43519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.65
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:17:23.127686 31351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:23.127879 31351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:17:23.133554 31351 server_base.cc:1034] running on GCE node
W20260810 07:17:23.133507 31358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:23.133525 31361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:23.133507 31359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:23.134037 31351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:23.134311 31351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:23.135489 31351 hybrid_clock.cc:648] HybridClock initialized: now 1786346243135474 us; error 33 us; skew 500 ppm
Aug 10 07:17:23 dist-test-slave-10pc krb5kdc[31230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346243, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:23.138815 31351 init.cc:377] Logged in from keytab as kudu/127.15.217.65@KRBTEST.COM (short username kudu)
I20260810 07:17:23.139297 31351 webserver.cc:460] Webserver started at http://127.15.217.65:46295/ using document root <none> and password file <none>
I20260810 07:17:23.139530 31351 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:23.139577 31351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:23.139679 31351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:17:23.140992 31351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/data/instance:
uuid: "d5f62e4899314ee5a13cff40ee0258a8"
format_stamp: "Formatted at 2026-08-10 07:17:23 on dist-test-slave-10pc"
server_key: "0172fd146aa36ef87b83957404b14952"
server_key_iv: "0f8c7fa7f6aba3d58ea8748dec8043e0"
server_key_version: "encryptionkey@0"
I20260810 07:17:23.141358 31351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance:
uuid: "d5f62e4899314ee5a13cff40ee0258a8"
format_stamp: "Formatted at 2026-08-10 07:17:23 on dist-test-slave-10pc"
server_key: "0172fd146aa36ef87b83957404b14952"
server_key_iv: "0f8c7fa7f6aba3d58ea8748dec8043e0"
server_key_version: "encryptionkey@0"
I20260810 07:17:23.142854 31351 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:17:23.143797 31369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:23.144013 31351 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:17:23.144111 31351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/wal
uuid: "d5f62e4899314ee5a13cff40ee0258a8"
format_stamp: "Formatted at 2026-08-10 07:17:23 on dist-test-slave-10pc"
server_key: "0172fd146aa36ef87b83957404b14952"
server_key_iv: "0f8c7fa7f6aba3d58ea8748dec8043e0"
server_key_version: "encryptionkey@0"
I20260810 07:17:23.144184 31351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:23.166252 31351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:23.168933 31351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:23.169114 31351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:23.169338 31351 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:17:23.169730 31351 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:17:23.169770 31351 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:23.169817 31351 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:17:23.169847 31351 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:23.176190 31351 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.65:41865
I20260810 07:17:23.176321 31489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.65:41865 every 8 connection(s)
I20260810 07:17:23.176676 31351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
Aug 10 07:17:23 dist-test-slave-10pc krb5kdc[31230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346243, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:17:23.185493 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 31351
I20260810 07:17:23.185655 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance
I20260810 07:17:23.185927 16229 external_mini_cluster.cc:1428] Setting key 2b58d73e408944d251a9bf5e2e9b6378
I20260810 07:17:23.187444 31492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:23.177485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.65:57469)
0810 07:17:23.177665 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:23.177711 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:23.178331 (+ 620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:23.178769 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:23.178774 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:23.179447 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:23.179734 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:23.179740 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.180435 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.180435 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:23.181316 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:23.181318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.181735 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.181736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:23.181849 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:23.182455 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:23.182465 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:23.183920 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:23.185759 (+ 1839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:23.185767 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:23.185771 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:23.186106 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:23.186378 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:23.186380 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:23.186380 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:23.186523 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:23.186912 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:23.186918 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:23.187053 (+ 135us) client_negotiation.cc:769] Sending connection context
0810 07:17:23.187143 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 07:17:23.187271 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":116,"threads_started":1}
I20260810 07:17:23.187481 31325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:23.177539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:57469 (local address 127.15.217.126:43519)
0810 07:17:23.177604 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:23.177606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:23.177846 (+ 240us) server_negotiation.cc:407] Connection header received
0810 07:17:23.178465 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:23.178583 (+ 118us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:23.178604 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:23.178682 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:23.179830 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.180346 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.181415 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.181650 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.181715 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:23.184015 (+ 2300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:23.184016 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:23.184017 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:23.184026 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:23.185640 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:23.186197 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:23.186199 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:23.186199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:23.186286 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:23.186612 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:23.186613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:23.186614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:23.186820 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:23.186901 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:23.187202 (+ 301us) server_negotiation.cc:299] Negotiation successful
0810 07:17:23.187327 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 07:17:23.188021 31490 heartbeater.cc:347] Connected to a master server at 127.15.217.126:43519
I20260810 07:17:23.188126 31490 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:23.188335 31490 heartbeater.cc:511] Master 127.15.217.126:43519 requested a full tablet report, sending...
I20260810 07:17:23.188999 31282 ts_manager.cc:194] Registered new tserver with Master: d5f62e4899314ee5a13cff40ee0258a8 (127.15.217.65:41865)
I20260810 07:17:23.189917 31282 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.65@KRBTEST.COM'} at 127.15.217.65:57469
WARNING: no policy specified for kudu/127.15.217.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab.
Entry for principal kudu/127.15.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab.
I20260810 07:17:23.215032 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.66:0
--local_ip_for_outbound_sockets=127.15.217.66
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:43519
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:38625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:17:23.301281 31495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:23.301509 31495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:23.301535 31495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:23.302670 31495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:23.302706 31495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:23.302732 31495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.66
I20260810 07:17:23.304169 31495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:38625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:43519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.66
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:17:23.304389 31495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:23.304695 31495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:17:23.310639 31495 server_base.cc:1034] running on GCE node
W20260810 07:17:23.310705 31505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:23.310600 31503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:23.310618 31502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:23.311118 31495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:23.311343 31495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:23.312546 31495 hybrid_clock.cc:648] HybridClock initialized: now 1786346243312524 us; error 30 us; skew 500 ppm
Aug 10 07:17:23 dist-test-slave-10pc krb5kdc[31230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346243, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:23.315200 31495 init.cc:377] Logged in from keytab as kudu/127.15.217.66@KRBTEST.COM (short username kudu)
I20260810 07:17:23.315701 31495 webserver.cc:460] Webserver started at http://127.15.217.66:37491/ using document root <none> and password file <none>
I20260810 07:17:23.315922 31495 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:23.315963 31495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:23.316058 31495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:17:23.317193 31495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/data/instance:
uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e"
format_stamp: "Formatted at 2026-08-10 07:17:23 on dist-test-slave-10pc"
server_key: "80b71bc5edab3db9076d626e42dce357"
server_key_iv: "0aae04404f6609f798dd288d95a80dd6"
server_key_version: "encryptionkey@0"
I20260810 07:17:23.317509 31495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance:
uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e"
format_stamp: "Formatted at 2026-08-10 07:17:23 on dist-test-slave-10pc"
server_key: "80b71bc5edab3db9076d626e42dce357"
server_key_iv: "0aae04404f6609f798dd288d95a80dd6"
server_key_version: "encryptionkey@0"
I20260810 07:17:23.318815 31495 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:17:23.319566 31512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:23.319762 31495 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:17:23.319854 31495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/wal
uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e"
format_stamp: "Formatted at 2026-08-10 07:17:23 on dist-test-slave-10pc"
server_key: "80b71bc5edab3db9076d626e42dce357"
server_key_iv: "0aae04404f6609f798dd288d95a80dd6"
server_key_version: "encryptionkey@0"
I20260810 07:17:23.319934 31495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:23.339757 31495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:23.342064 31495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:23.342203 31495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:23.342417 31495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:17:23.342783 31495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:17:23.342818 31495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:23.342840 31495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:17:23.342890 31495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:23.349105 31495 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.66:44833
I20260810 07:17:23.349188 31633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.66:44833 every 8 connection(s)
I20260810 07:17:23.349546 31495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
I20260810 07:17:23.351070 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 31495
I20260810 07:17:23.351212 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance
I20260810 07:17:23.351436 16229 external_mini_cluster.cc:1428] Setting key aa9d31efc78117932d47484468f6c97d
Aug 10 07:17:23 dist-test-slave-10pc krb5kdc[31230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346243, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:17:23.360095 31325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:23.350351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:54839 (local address 127.15.217.126:43519)
0810 07:17:23.350418 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:23.350420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:23.350649 (+ 229us) server_negotiation.cc:407] Connection header received
0810 07:17:23.351346 (+ 697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:23.351347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:23.351361 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:23.351408 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:23.352613 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.353117 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.354344 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.354588 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.354653 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:23.357135 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:23.357136 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:23.357136 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:23.357144 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:23.358572 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:23.359090 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:23.359091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:23.359091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:23.359159 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:23.359458 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:23.359458 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:23.359458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:23.359561 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:23.359610 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:23.359958 (+ 348us) server_negotiation.cc:299] Negotiation successful
0810 07:17:23.360026 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 07:17:23.360410 31636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:23.350322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.66:54839)
0810 07:17:23.350524 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:23.350563 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:23.351222 (+ 659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:23.351487 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:23.351491 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:23.352252 (+ 761us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:23.352502 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:23.352509 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.353221 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.353223 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:23.354249 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:23.354251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.354677 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.354678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:23.354841 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:23.355379 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:23.355388 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:23.357014 (+ 1626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:23.358660 (+ 1646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:23.358669 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:23.358672 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:23.359011 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:23.359228 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:23.359229 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:23.359230 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:23.359386 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:23.359631 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:23.359638 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:23.359815 (+ 177us) client_negotiation.cc:769] Sending connection context
0810 07:17:23.359941 (+ 126us) client_negotiation.cc:240] Negotiation successful
0810 07:17:23.360156 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":109,"threads_started":1}
I20260810 07:17:23.361008 31634 heartbeater.cc:347] Connected to a master server at 127.15.217.126:43519
I20260810 07:17:23.361135 31634 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:23.361333 31634 heartbeater.cc:511] Master 127.15.217.126:43519 requested a full tablet report, sending...
I20260810 07:17:23.361792 31282 ts_manager.cc:194] Registered new tserver with Master: cab9a5eac3834eb6a7cfee1dbfcdad6e (127.15.217.66:44833)
I20260810 07:17:23.362190 31282 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.66@KRBTEST.COM'} at 127.15.217.66:54839
WARNING: no policy specified for kudu/127.15.217.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab.
Entry for principal kudu/127.15.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab.
I20260810 07:17:23.383530 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.67:0
--local_ip_for_outbound_sockets=127.15.217.67
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:43519
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:38625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:17:23.471154 31639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:23.471383 31639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:23.471406 31639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:23.472549 31639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:23.472585 31639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:23.472611 31639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.67
I20260810 07:17:23.474102 31639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:38625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:43519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.67
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:17:23.474355 31639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:23.474602 31639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:23.479893 31648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:23.479892 31645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:23.479934 31646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:23.479956 31639 server_base.cc:1034] running on GCE node
I20260810 07:17:23.480297 31639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:23.480584 31639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:23.481740 31639 hybrid_clock.cc:648] HybridClock initialized: now 1786346243481727 us; error 26 us; skew 500 ppm
Aug 10 07:17:23 dist-test-slave-10pc krb5kdc[31230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346243, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:23.484411 31639 init.cc:377] Logged in from keytab as kudu/127.15.217.67@KRBTEST.COM (short username kudu)
I20260810 07:17:23.484997 31639 webserver.cc:460] Webserver started at http://127.15.217.67:42763/ using document root <none> and password file <none>
I20260810 07:17:23.485245 31639 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:23.485316 31639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:23.485472 31639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:17:23.486598 31639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/data/instance:
uuid: "a8e2d66b94934297b6e280649a56d5df"
format_stamp: "Formatted at 2026-08-10 07:17:23 on dist-test-slave-10pc"
server_key: "53dd8717f0ca9e0bcd731240e692671e"
server_key_iv: "4c9a876b543b9205f1fddf19f014b87d"
server_key_version: "encryptionkey@0"
I20260810 07:17:23.486986 31639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance:
uuid: "a8e2d66b94934297b6e280649a56d5df"
format_stamp: "Formatted at 2026-08-10 07:17:23 on dist-test-slave-10pc"
server_key: "53dd8717f0ca9e0bcd731240e692671e"
server_key_iv: "4c9a876b543b9205f1fddf19f014b87d"
server_key_version: "encryptionkey@0"
I20260810 07:17:23.488361 31639 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:17:23.489199 31656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:23.489379 31639 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:17:23.489503 31639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/wal
uuid: "a8e2d66b94934297b6e280649a56d5df"
format_stamp: "Formatted at 2026-08-10 07:17:23 on dist-test-slave-10pc"
server_key: "53dd8717f0ca9e0bcd731240e692671e"
server_key_iv: "4c9a876b543b9205f1fddf19f014b87d"
server_key_version: "encryptionkey@0"
I20260810 07:17:23.489584 31639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:23.504519 31639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:23.506618 31639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:23.506781 31639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:23.507033 31639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:17:23.507362 31639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:17:23.507418 31639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:23.507463 31639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:17:23.507488 31639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:23.513846 31639 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.67:33891
I20260810 07:17:23.513900 31784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.67:33891 every 8 connection(s)
I20260810 07:17:23.514221 31639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
I20260810 07:17:23.520061 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 31639
I20260810 07:17:23.520156 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance
I20260810 07:17:23.520345 16229 external_mini_cluster.cc:1428] Setting key 79f7ad3ddae0b421e759386accb84d34
Aug 10 07:17:23 dist-test-slave-10pc krb5kdc[31230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346243, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:17:23.524111 31325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:23.514973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:40661 (local address 127.15.217.126:43519)
0810 07:17:23.515070 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:23.515073 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:23.515282 (+ 209us) server_negotiation.cc:407] Connection header received
0810 07:17:23.515780 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:23.515781 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:23.515790 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:23.515850 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:23.516895 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.517406 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.518413 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.518598 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.518651 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:23.521184 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:23.521185 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:23.521186 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:23.521193 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:23.522489 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:23.523057 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:23.523059 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:23.523059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:23.523131 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:23.523395 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:23.523396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:23.523397 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:23.523546 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:23.523628 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:23.523930 (+ 302us) server_negotiation.cc:299] Negotiation successful
0810 07:17:23.524019 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 07:17:23.524324 31787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:23.514929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.67:40661)
0810 07:17:23.515143 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:23.515188 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:23.515696 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:23.515907 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:23.515910 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:23.516585 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:23.516802 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:23.516807 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.517494 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.517494 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:23.518332 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:23.518334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.518673 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.518674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:23.518784 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:23.519358 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:23.519367 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:23.521094 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:23.522670 (+ 1576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:23.522679 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:23.522681 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:23.522968 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:23.523206 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:23.523207 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:23.523207 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:23.523324 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:23.523617 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:23.523626 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:23.523774 (+ 148us) client_negotiation.cc:769] Sending connection context
0810 07:17:23.523889 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 07:17:23.524068 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":94,"threads_started":1}
I20260810 07:17:23.524825 31785 heartbeater.cc:347] Connected to a master server at 127.15.217.126:43519
I20260810 07:17:23.524953 31785 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:23.525168 31785 heartbeater.cc:511] Master 127.15.217.126:43519 requested a full tablet report, sending...
I20260810 07:17:23.525578 31282 ts_manager.cc:194] Registered new tserver with Master: a8e2d66b94934297b6e280649a56d5df (127.15.217.67:33891)
I20260810 07:17:23.525971 31282 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.67@KRBTEST.COM'} at 127.15.217.67:40661
I20260810 07:17:23.532960 16229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:17:23.541188 31325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:23.534464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39894 (local address 127.15.217.126:43519)
0810 07:17:23.534533 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:23.534536 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:23.534737 (+ 201us) server_negotiation.cc:407] Connection header received
0810 07:17:23.534767 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:23.534768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:23.534779 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:23.534843 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:23.535421 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.536016 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.536794 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.536993 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.537066 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:23.538371 (+ 1305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:23.538373 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:23.538373 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:23.538381 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:23.539831 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:23.540173 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:23.540174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:23.540174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:23.540234 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:23.540421 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:23.540421 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:23.540422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:23.540737 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:23.540822 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:23.541045 (+ 223us) server_negotiation.cc:299] Negotiation successful
0810 07:17:23.541119 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 07:17:23.543190 31282 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39894:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:17:23.544997 31282 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:17:23.552021 31802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:23.547794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:33891 (local address 127.0.0.1:49214)
0810 07:17:23.548246 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:23.548255 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:23.548309 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:23.548599 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:23.548600 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:23.548602 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:17:23.548640 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:23.548641 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.549508 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.549509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:23.550719 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:23.550722 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.551707 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.551708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:23.551796 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:23.551814 (+ 18us) client_negotiation.cc:769] Sending connection context
0810 07:17:23.551882 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 07:17:23.551934 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":79,"threads_started":1}
I20260810 07:17:23.552148 31803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:23.547947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60470 (local address 127.15.217.65:41865)
0810 07:17:23.548143 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:23.548149 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:23.548162 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:17:23.548205 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:23.548207 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:23.548225 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:23.548305 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:17:23.548639 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.549799 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.551010 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.551740 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.551837 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:23.551894 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:23.552015 (+ 121us) server_negotiation.cc:299] Negotiation successful
0810 07:17:23.552064 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":113,"threads_started":1}
I20260810 07:17:23.552352 31804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:23.548050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49214 (local address 127.15.217.67:33891)
0810 07:17:23.548225 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:23.548229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:23.548363 (+ 134us) server_negotiation.cc:407] Connection header received
0810 07:17:23.548406 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:23.548408 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:23.548424 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:23.548516 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:17:23.548702 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.549403 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.550825 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.551606 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.551689 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:23.551757 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:23.552219 (+ 462us) server_negotiation.cc:299] Negotiation successful
0810 07:17:23.552271 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":100,"threads_started":1}
I20260810 07:17:23.552564 31798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:23.547645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:41865 (local address 127.0.0.1:60470)
0810 07:17:23.547803 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:23.547850 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:23.547917 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:23.548326 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:23.548331 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:23.548333 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:17:23.548411 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:23.548413 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.549916 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.549917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:23.550932 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:23.550934 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.551848 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.551849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:23.551929 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:23.551933 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:17:23.551991 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 07:17:23.552035 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":97,"threads_started":1}
I20260810 07:17:23.553054 31800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:23.547730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34660 (local address 127.15.217.66:44833)
0810 07:17:23.548207 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:23.548211 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:23.548220 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:17:23.548260 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:23.548261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:23.548277 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:23.548585 (+ 308us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:17:23.548681 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.549798 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.551758 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.552559 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.552889 (+ 330us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:23.552941 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:23.552961 (+ 20us) server_negotiation.cc:299] Negotiation successful
0810 07:17:23.552993 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":89,"threads_started":1}
I20260810 07:17:23.553143 31712 tablet_service.cc:1467] Processing CreateTablet for tablet a55a1bce75a34d0c8cf23bc4e798832b (DEFAULT_TABLE table=test-table [id=e10211a606dc4f54ba4c7bfcd57a1682]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:17:23.552834 31799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:23.547585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:44833 (local address 127.0.0.1:34660)
0810 07:17:23.547801 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:23.547848 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:23.547917 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:23.548383 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:23.548384 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:23.548386 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:17:23.548456 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:23.548458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.551001 (+ 2543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.551001 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:23.551643 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:23.551644 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.552662 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.552663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:23.552732 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:23.552735 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:17:23.552760 (+ 25us) client_negotiation.cc:240] Negotiation successful
0810 07:17:23.552790 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":66,"threads_started":1}
I20260810 07:17:23.553642 31712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a55a1bce75a34d0c8cf23bc4e798832b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:17:23.553829 31421 tablet_service.cc:1467] Processing CreateTablet for tablet a55a1bce75a34d0c8cf23bc4e798832b (DEFAULT_TABLE table=test-table [id=e10211a606dc4f54ba4c7bfcd57a1682]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:17:23.554026 31565 tablet_service.cc:1467] Processing CreateTablet for tablet a55a1bce75a34d0c8cf23bc4e798832b (DEFAULT_TABLE table=test-table [id=e10211a606dc4f54ba4c7bfcd57a1682]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:17:23.554165 31421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a55a1bce75a34d0c8cf23bc4e798832b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:17:23.554373 31565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a55a1bce75a34d0c8cf23bc4e798832b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:17:23.556896 31805 tablet_bootstrap.cc:492] T a55a1bce75a34d0c8cf23bc4e798832b P d5f62e4899314ee5a13cff40ee0258a8: Bootstrap starting.
I20260810 07:17:23.557361 31806 tablet_bootstrap.cc:492] T a55a1bce75a34d0c8cf23bc4e798832b P a8e2d66b94934297b6e280649a56d5df: Bootstrap starting.
I20260810 07:17:23.557628 31805 tablet_bootstrap.cc:654] T a55a1bce75a34d0c8cf23bc4e798832b P d5f62e4899314ee5a13cff40ee0258a8: Neither blocks nor log segments found. Creating new log.
I20260810 07:17:23.558136 31806 tablet_bootstrap.cc:654] T a55a1bce75a34d0c8cf23bc4e798832b P a8e2d66b94934297b6e280649a56d5df: Neither blocks nor log segments found. Creating new log.
I20260810 07:17:23.558156 31807 tablet_bootstrap.cc:492] T a55a1bce75a34d0c8cf23bc4e798832b P cab9a5eac3834eb6a7cfee1dbfcdad6e: Bootstrap starting.
I20260810 07:17:23.558355 31805 log.cc:824] T a55a1bce75a34d0c8cf23bc4e798832b P d5f62e4899314ee5a13cff40ee0258a8: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:23.558890 31806 log.cc:824] T a55a1bce75a34d0c8cf23bc4e798832b P a8e2d66b94934297b6e280649a56d5df: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:23.559017 31807 tablet_bootstrap.cc:654] T a55a1bce75a34d0c8cf23bc4e798832b P cab9a5eac3834eb6a7cfee1dbfcdad6e: Neither blocks nor log segments found. Creating new log.
I20260810 07:17:23.559363 31805 tablet_bootstrap.cc:492] T a55a1bce75a34d0c8cf23bc4e798832b P d5f62e4899314ee5a13cff40ee0258a8: No bootstrap required, opened a new log
I20260810 07:17:23.559461 31805 ts_tablet_manager.cc:1397] T a55a1bce75a34d0c8cf23bc4e798832b P d5f62e4899314ee5a13cff40ee0258a8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:17:23.559617 31807 log.cc:824] T a55a1bce75a34d0c8cf23bc4e798832b P cab9a5eac3834eb6a7cfee1dbfcdad6e: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:23.559670 31806 tablet_bootstrap.cc:492] T a55a1bce75a34d0c8cf23bc4e798832b P a8e2d66b94934297b6e280649a56d5df: No bootstrap required, opened a new log
I20260810 07:17:23.559760 31806 ts_tablet_manager.cc:1397] T a55a1bce75a34d0c8cf23bc4e798832b P a8e2d66b94934297b6e280649a56d5df: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:17:23.560328 31807 tablet_bootstrap.cc:492] T a55a1bce75a34d0c8cf23bc4e798832b P cab9a5eac3834eb6a7cfee1dbfcdad6e: No bootstrap required, opened a new log
I20260810 07:17:23.560397 31807 ts_tablet_manager.cc:1397] T a55a1bce75a34d0c8cf23bc4e798832b P cab9a5eac3834eb6a7cfee1dbfcdad6e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:17:23.561456 31806 raft_consensus.cc:348] T a55a1bce75a34d0c8cf23bc4e798832b P a8e2d66b94934297b6e280649a56d5df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44833 } } peers { permanent_uuid: "a8e2d66b94934297b6e280649a56d5df" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 33891 } } peers { permanent_uuid: "d5f62e4899314ee5a13cff40ee0258a8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41865 } }
I20260810 07:17:23.561633 31806 raft_consensus.cc:374] T a55a1bce75a34d0c8cf23bc4e798832b P a8e2d66b94934297b6e280649a56d5df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:17:23.561679 31806 raft_consensus.cc:729] T a55a1bce75a34d0c8cf23bc4e798832b P a8e2d66b94934297b6e280649a56d5df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8e2d66b94934297b6e280649a56d5df, State: Initialized, Role: FOLLOWER
I20260810 07:17:23.561792 31806 consensus_queue.cc:260] T a55a1bce75a34d0c8cf23bc4e798832b P a8e2d66b94934297b6e280649a56d5df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44833 } } peers { permanent_uuid: "a8e2d66b94934297b6e280649a56d5df" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 33891 } } peers { permanent_uuid: "d5f62e4899314ee5a13cff40ee0258a8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41865 } }
I20260810 07:17:23.561764 31805 raft_consensus.cc:348] T a55a1bce75a34d0c8cf23bc4e798832b P d5f62e4899314ee5a13cff40ee0258a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44833 } } peers { permanent_uuid: "a8e2d66b94934297b6e280649a56d5df" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 33891 } } peers { permanent_uuid: "d5f62e4899314ee5a13cff40ee0258a8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41865 } }
I20260810 07:17:23.561932 31805 raft_consensus.cc:374] T a55a1bce75a34d0c8cf23bc4e798832b P d5f62e4899314ee5a13cff40ee0258a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:17:23.561968 31805 raft_consensus.cc:729] T a55a1bce75a34d0c8cf23bc4e798832b P d5f62e4899314ee5a13cff40ee0258a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5f62e4899314ee5a13cff40ee0258a8, State: Initialized, Role: FOLLOWER
I20260810 07:17:23.562084 31805 consensus_queue.cc:260] T a55a1bce75a34d0c8cf23bc4e798832b P d5f62e4899314ee5a13cff40ee0258a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44833 } } peers { permanent_uuid: "a8e2d66b94934297b6e280649a56d5df" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 33891 } } peers { permanent_uuid: "d5f62e4899314ee5a13cff40ee0258a8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41865 } }
I20260810 07:17:23.562136 31806 ts_tablet_manager.cc:1428] T a55a1bce75a34d0c8cf23bc4e798832b P a8e2d66b94934297b6e280649a56d5df: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 07:17:23.562292 31785 heartbeater.cc:503] Master 127.15.217.126:43519 was elected leader, sending a full tablet report...
I20260810 07:17:23.562397 31805 ts_tablet_manager.cc:1428] T a55a1bce75a34d0c8cf23bc4e798832b P d5f62e4899314ee5a13cff40ee0258a8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:17:23.562582 31490 heartbeater.cc:503] Master 127.15.217.126:43519 was elected leader, sending a full tablet report...
I20260810 07:17:23.562479 31807 raft_consensus.cc:348] T a55a1bce75a34d0c8cf23bc4e798832b P cab9a5eac3834eb6a7cfee1dbfcdad6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44833 } } peers { permanent_uuid: "a8e2d66b94934297b6e280649a56d5df" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 33891 } } peers { permanent_uuid: "d5f62e4899314ee5a13cff40ee0258a8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41865 } }
I20260810 07:17:23.562690 31807 raft_consensus.cc:374] T a55a1bce75a34d0c8cf23bc4e798832b P cab9a5eac3834eb6a7cfee1dbfcdad6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:17:23.562742 31807 raft_consensus.cc:729] T a55a1bce75a34d0c8cf23bc4e798832b P cab9a5eac3834eb6a7cfee1dbfcdad6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cab9a5eac3834eb6a7cfee1dbfcdad6e, State: Initialized, Role: FOLLOWER
I20260810 07:17:23.562849 31807 consensus_queue.cc:260] T a55a1bce75a34d0c8cf23bc4e798832b P cab9a5eac3834eb6a7cfee1dbfcdad6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44833 } } peers { permanent_uuid: "a8e2d66b94934297b6e280649a56d5df" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 33891 } } peers { permanent_uuid: "d5f62e4899314ee5a13cff40ee0258a8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41865 } }
I20260810 07:17:23.563184 31634 heartbeater.cc:503] Master 127.15.217.126:43519 was elected leader, sending a full tablet report...
I20260810 07:17:23.563423 31807 ts_tablet_manager.cc:1428] T a55a1bce75a34d0c8cf23bc4e798832b P cab9a5eac3834eb6a7cfee1dbfcdad6e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:17:23.569245 31813 raft_consensus.cc:482] T a55a1bce75a34d0c8cf23bc4e798832b P cab9a5eac3834eb6a7cfee1dbfcdad6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:17:23.569396 31813 raft_consensus.cc:504] T a55a1bce75a34d0c8cf23bc4e798832b P cab9a5eac3834eb6a7cfee1dbfcdad6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44833 } } peers { permanent_uuid: "a8e2d66b94934297b6e280649a56d5df" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 33891 } } peers { permanent_uuid: "d5f62e4899314ee5a13cff40ee0258a8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41865 } }
I20260810 07:17:23.569883 31813 leader_election.cc:288] T a55a1bce75a34d0c8cf23bc4e798832b P cab9a5eac3834eb6a7cfee1dbfcdad6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8e2d66b94934297b6e280649a56d5df (127.15.217.67:33891), d5f62e4899314ee5a13cff40ee0258a8 (127.15.217.65:41865)
I20260810 07:17:23.572803 31636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:23.570101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:41865 (local address 127.15.217.66:46745)
0810 07:17:23.570161 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:23.570174 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:23.570225 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:23.570365 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:23.570366 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:23.570368 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:17:23.570437 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:23.570439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.571133 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.571134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:23.571890 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:23.571892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.572501 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.572502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:23.572594 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:23.572598 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:17:23.572677 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 07:17:23.572721 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 07:17:23.572803 31803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:23.570173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:46745 (local address 127.15.217.65:41865)
0810 07:17:23.570224 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:23.570226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:23.570236 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:17:23.570288 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:23.570289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:23.570300 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:23.570346 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:17:23.570515 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.571009 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.571995 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.572377 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.572465 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:23.572504 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:23.572688 (+ 184us) server_negotiation.cc:299] Negotiation successful
0810 07:17:23.572731 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 07:17:23.572928 31804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:23.570280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:55069 (local address 127.15.217.67:33891)
0810 07:17:23.570326 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:23.570328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:23.570441 (+ 113us) server_negotiation.cc:407] Connection header received
0810 07:17:23.570512 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:23.570513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:23.570522 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:23.570572 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:17:23.570728 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.571202 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.572127 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.572582 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.572666 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:23.572674 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:23.572825 (+ 151us) server_negotiation.cc:299] Negotiation successful
0810 07:17:23.572865 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 07:17:23.572939 31815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:23.570163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:33891 (local address 127.15.217.66:55069)
0810 07:17:23.570368 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:23.570377 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:23.570453 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:23.570573 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:23.570573 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:23.570575 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:17:23.570644 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:23.570645 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.571287 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.571288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:23.572025 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:23.572027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.572667 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.572668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:23.572741 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:23.572744 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:17:23.572819 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 07:17:23.572859 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":112,"threads_started":1}
I20260810 07:17:23.573220 31443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a55a1bce75a34d0c8cf23bc4e798832b" candidate_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5f62e4899314ee5a13cff40ee0258a8" is_pre_election: true
I20260810 07:17:23.573379 31443 raft_consensus.cc:2454] T a55a1bce75a34d0c8cf23bc4e798832b P d5f62e4899314ee5a13cff40ee0258a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cab9a5eac3834eb6a7cfee1dbfcdad6e in term 0.
I20260810 07:17:23.573339 31733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a55a1bce75a34d0c8cf23bc4e798832b" candidate_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8e2d66b94934297b6e280649a56d5df" is_pre_election: true
I20260810 07:17:23.573534 31733 raft_consensus.cc:2454] T a55a1bce75a34d0c8cf23bc4e798832b P a8e2d66b94934297b6e280649a56d5df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cab9a5eac3834eb6a7cfee1dbfcdad6e in term 0.
I20260810 07:17:23.573653 31514 leader_election.cc:302] T a55a1bce75a34d0c8cf23bc4e798832b P cab9a5eac3834eb6a7cfee1dbfcdad6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cab9a5eac3834eb6a7cfee1dbfcdad6e, d5f62e4899314ee5a13cff40ee0258a8; no voters:
I20260810 07:17:23.573804 31813 raft_consensus.cc:2781] T a55a1bce75a34d0c8cf23bc4e798832b P cab9a5eac3834eb6a7cfee1dbfcdad6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:17:23.573874 31813 raft_consensus.cc:482] T a55a1bce75a34d0c8cf23bc4e798832b P cab9a5eac3834eb6a7cfee1dbfcdad6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:17:23.573897 31813 raft_consensus.cc:3037] T a55a1bce75a34d0c8cf23bc4e798832b P cab9a5eac3834eb6a7cfee1dbfcdad6e [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:17:23.574676 31813 raft_consensus.cc:504] T a55a1bce75a34d0c8cf23bc4e798832b P cab9a5eac3834eb6a7cfee1dbfcdad6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44833 } } peers { permanent_uuid: "a8e2d66b94934297b6e280649a56d5df" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 33891 } } peers { permanent_uuid: "d5f62e4899314ee5a13cff40ee0258a8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41865 } }
I20260810 07:17:23.574858 31813 leader_election.cc:288] T a55a1bce75a34d0c8cf23bc4e798832b P cab9a5eac3834eb6a7cfee1dbfcdad6e [CANDIDATE]: Term 1 election: Requested vote from peers a8e2d66b94934297b6e280649a56d5df (127.15.217.67:33891), d5f62e4899314ee5a13cff40ee0258a8 (127.15.217.65:41865)
I20260810 07:17:23.574980 31733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a55a1bce75a34d0c8cf23bc4e798832b" candidate_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8e2d66b94934297b6e280649a56d5df"
I20260810 07:17:23.575065 31733 raft_consensus.cc:3037] T a55a1bce75a34d0c8cf23bc4e798832b P a8e2d66b94934297b6e280649a56d5df [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:17:23.575038 31443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a55a1bce75a34d0c8cf23bc4e798832b" candidate_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5f62e4899314ee5a13cff40ee0258a8"
I20260810 07:17:23.575155 31443 raft_consensus.cc:3037] T a55a1bce75a34d0c8cf23bc4e798832b P d5f62e4899314ee5a13cff40ee0258a8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:17:23.575870 31733 raft_consensus.cc:2454] T a55a1bce75a34d0c8cf23bc4e798832b P a8e2d66b94934297b6e280649a56d5df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cab9a5eac3834eb6a7cfee1dbfcdad6e in term 1.
I20260810 07:17:23.576122 31513 leader_election.cc:302] T a55a1bce75a34d0c8cf23bc4e798832b P cab9a5eac3834eb6a7cfee1dbfcdad6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8e2d66b94934297b6e280649a56d5df, cab9a5eac3834eb6a7cfee1dbfcdad6e; no voters:
I20260810 07:17:23.576116 31443 raft_consensus.cc:2454] T a55a1bce75a34d0c8cf23bc4e798832b P d5f62e4899314ee5a13cff40ee0258a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cab9a5eac3834eb6a7cfee1dbfcdad6e in term 1.
I20260810 07:17:23.576220 31813 raft_consensus.cc:2781] T a55a1bce75a34d0c8cf23bc4e798832b P cab9a5eac3834eb6a7cfee1dbfcdad6e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:17:23.576393 31813 raft_consensus.cc:686] T a55a1bce75a34d0c8cf23bc4e798832b P cab9a5eac3834eb6a7cfee1dbfcdad6e [term 1 LEADER]: Becoming Leader. State: Replica: cab9a5eac3834eb6a7cfee1dbfcdad6e, State: Running, Role: LEADER
I20260810 07:17:23.576537 31813 consensus_queue.cc:237] T a55a1bce75a34d0c8cf23bc4e798832b P cab9a5eac3834eb6a7cfee1dbfcdad6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44833 } } peers { permanent_uuid: "a8e2d66b94934297b6e280649a56d5df" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 33891 } } peers { permanent_uuid: "d5f62e4899314ee5a13cff40ee0258a8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41865 } }
I20260810 07:17:23.577447 31282 catalog_manager.cc:5495] T a55a1bce75a34d0c8cf23bc4e798832b P cab9a5eac3834eb6a7cfee1dbfcdad6e reported cstate change: term changed from 0 to 1, leader changed from <none> to cab9a5eac3834eb6a7cfee1dbfcdad6e (127.15.217.66). New cstate: current_term: 1 leader_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44833 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8e2d66b94934297b6e280649a56d5df" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 33891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5f62e4899314ee5a13cff40ee0258a8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41865 } health_report { overall_health: UNKNOWN } } }
I20260810 07:17:23.588696 31800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:23.586118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34672 (local address 127.15.217.66:44833)
0810 07:17:23.586167 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:23.586169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:23.586179 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:17:23.586260 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:23.586261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:23.586287 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:23.586363 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:17:23.586514 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.586999 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.587827 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.588037 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.588107 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:23.588293 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:17:23.588401 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:17:23.588535 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:23.588554 (+ 19us) server_negotiation.cc:299] Negotiation successful
0810 07:17:23.588575 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 07:17:23.590583 31443 raft_consensus.cc:1261] T a55a1bce75a34d0c8cf23bc4e798832b P d5f62e4899314ee5a13cff40ee0258a8 [term 1 FOLLOWER]: Refusing update from remote peer cab9a5eac3834eb6a7cfee1dbfcdad6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:17:23.590584 31733 raft_consensus.cc:1261] T a55a1bce75a34d0c8cf23bc4e798832b P a8e2d66b94934297b6e280649a56d5df [term 1 FOLLOWER]: Refusing update from remote peer cab9a5eac3834eb6a7cfee1dbfcdad6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:17:23.591009 31817 consensus_queue.cc:1035] T a55a1bce75a34d0c8cf23bc4e798832b P cab9a5eac3834eb6a7cfee1dbfcdad6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5f62e4899314ee5a13cff40ee0258a8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:17:23.591164 31813 consensus_queue.cc:1035] T a55a1bce75a34d0c8cf23bc4e798832b P cab9a5eac3834eb6a7cfee1dbfcdad6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8e2d66b94934297b6e280649a56d5df" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 33891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:17:23.593742 31821 mvcc.cc:205] Tried to move back new op lower bound from 7316874213744066560 to 7316874213690372096. Current Snapshot: MvccSnapshot[applied={T|T < 7316874213744066560}]
I20260810 07:17:23.594708 31823 mvcc.cc:205] Tried to move back new op lower bound from 7316874213744066560 to 7316874213690372096. Current Snapshot: MvccSnapshot[applied={T|T < 7316874213744066560 or (T in {7316874213744066560})}]
I20260810 07:17:23.596524 31281 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:17:23.596638 31281 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:17:23.597471 31281 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3: Sending DeleteTablet for 3 replicas of tablet a55a1bce75a34d0c8cf23bc4e798832b
I20260810 07:17:23.597852 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 31351
I20260810 07:17:23.597856 31421 tablet_service.cc:1514] Processing DeleteTablet for tablet a55a1bce75a34d0c8cf23bc4e798832b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:17:23 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:60470
I20260810 07:17:23.597914 31565 tablet_service.cc:1514] Processing DeleteTablet for tablet a55a1bce75a34d0c8cf23bc4e798832b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:17:23 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:34660
I20260810 07:17:23.598145 31712 tablet_service.cc:1514] Processing DeleteTablet for tablet a55a1bce75a34d0c8cf23bc4e798832b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:17:23 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:49214
I20260810 07:17:23.601678 31832 tablet_replica.cc:330] T a55a1bce75a34d0c8cf23bc4e798832b P cab9a5eac3834eb6a7cfee1dbfcdad6e: stopping tablet replica
I20260810 07:17:23.601797 31832 raft_consensus.cc:2229] T a55a1bce75a34d0c8cf23bc4e798832b P cab9a5eac3834eb6a7cfee1dbfcdad6e [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:17:23.601905 31832 raft_consensus.cc:2258] T a55a1bce75a34d0c8cf23bc4e798832b P cab9a5eac3834eb6a7cfee1dbfcdad6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:17:23.602391 31832 ts_tablet_manager.cc:1905] T a55a1bce75a34d0c8cf23bc4e798832b P cab9a5eac3834eb6a7cfee1dbfcdad6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:17:23.604274 31832 ts_tablet_manager.cc:1918] T a55a1bce75a34d0c8cf23bc4e798832b P cab9a5eac3834eb6a7cfee1dbfcdad6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:17:23.604357 31832 log.cc:1196] T a55a1bce75a34d0c8cf23bc4e798832b P cab9a5eac3834eb6a7cfee1dbfcdad6e: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/wal/wals/a55a1bce75a34d0c8cf23bc4e798832b
I20260810 07:17:23.604688 31832 ts_tablet_manager.cc:1939] T a55a1bce75a34d0c8cf23bc4e798832b P cab9a5eac3834eb6a7cfee1dbfcdad6e: Deleting consensus metadata
I20260810 07:17:23.605207 31264 catalog_manager.cc:4841] TS cab9a5eac3834eb6a7cfee1dbfcdad6e (127.15.217.66:44833): tablet a55a1bce75a34d0c8cf23bc4e798832b (table test-table [id=e10211a606dc4f54ba4c7bfcd57a1682]) successfully deleted
W20260810 07:17:23.608606 31265 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.217.65:41865 (error 108)
I20260810 07:17:23.608771 31833 tablet_replica.cc:330] T a55a1bce75a34d0c8cf23bc4e798832b P a8e2d66b94934297b6e280649a56d5df: stopping tablet replica
I20260810 07:17:23.608981 31833 raft_consensus.cc:2229] T a55a1bce75a34d0c8cf23bc4e798832b P a8e2d66b94934297b6e280649a56d5df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:17:23.609172 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 31495
I20260810 07:17:23.609292 31798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:23.609160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:41865 (local address 127.0.0.1:60480)
0810 07:17:23.609218 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:23.609255 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.65:41865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 07:17:23.610153 31265 catalog_manager.cc:4584] TS d5f62e4899314ee5a13cff40ee0258a8 (127.15.217.65:41865): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a55a1bce75a34d0c8cf23bc4e798832b: Network error: Client connection negotiation failed: client connection to 127.15.217.65:41865: connect: Connection refused (error 111)
I20260810 07:17:23.609259 31833 raft_consensus.cc:2258] T a55a1bce75a34d0c8cf23bc4e798832b P a8e2d66b94934297b6e280649a56d5df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:17:23.610987 31833 ts_tablet_manager.cc:1905] T a55a1bce75a34d0c8cf23bc4e798832b P a8e2d66b94934297b6e280649a56d5df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:17:23.612648 31833 ts_tablet_manager.cc:1918] T a55a1bce75a34d0c8cf23bc4e798832b P a8e2d66b94934297b6e280649a56d5df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:17:23.612743 31833 log.cc:1196] T a55a1bce75a34d0c8cf23bc4e798832b P a8e2d66b94934297b6e280649a56d5df: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/wal/wals/a55a1bce75a34d0c8cf23bc4e798832b
I20260810 07:17:23.613009 31833 ts_tablet_manager.cc:1939] T a55a1bce75a34d0c8cf23bc4e798832b P a8e2d66b94934297b6e280649a56d5df: Deleting consensus metadata
I20260810 07:17:23.613477 31264 catalog_manager.cc:4841] TS a8e2d66b94934297b6e280649a56d5df (127.15.217.67:33891): tablet a55a1bce75a34d0c8cf23bc4e798832b (table test-table [id=e10211a606dc4f54ba4c7bfcd57a1682]) successfully deleted
I20260810 07:17:23.617065 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 31639
I20260810 07:17:23.623535 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 31245
I20260810 07:17:23.631166 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:43519
--webserver_interface=127.15.217.126
--webserver_port=43441
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:38625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:43519
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:17:23.719383 31836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:23.719599 31836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:23.719622 31836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:23.720862 31836 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:17:23.720907 31836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:23.720928 31836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:23.720942 31836 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:17:23.721000 31836 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:17:23.722535 31836 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:38625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:43519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:43519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=43441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:17:23.722818 31836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:23.723089 31836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:23.733753 31842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:23.733753 31843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:23.733753 31846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:23.733788 31836 server_base.cc:1034] running on GCE node
I20260810 07:17:23.734349 31836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:23.734743 31836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:23.735926 31836 hybrid_clock.cc:648] HybridClock initialized: now 1786346243735916 us; error 39 us; skew 500 ppm
Aug 10 07:17:23 dist-test-slave-10pc krb5kdc[31230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346243, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:23.743821 31836 init.cc:377] Logged in from keytab as kudu/127.15.217.126@KRBTEST.COM (short username kudu)
I20260810 07:17:23.744364 31836 webserver.cc:460] Webserver started at http://127.15.217.126:43441/ using document root <none> and password file <none>
I20260810 07:17:23.744696 31836 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:23.744788 31836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:23.746377 31836 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:17:23.747176 31854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:23.747380 31836 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:17:23.747494 31836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "b5b017740ffd4355814fc558cbe98de3"
format_stamp: "Formatted at 2026-08-10 07:17:22 on dist-test-slave-10pc"
server_key: "9cf669d801a5ea9d14511819eaf4ab2b"
server_key_iv: "959cdcfc8ac39d4c29d7d9f1a271b1c8"
server_key_version: "encryptionkey@0"
I20260810 07:17:23.747978 31836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:23.764278 31836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:23.766923 31836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:23.767114 31836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:23.771845 31836 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:43519
I20260810 07:17:23.771898 31915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:43519 every 8 connection(s)
I20260810 07:17:23.772348 31836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:17:23.774196 31916 sys_catalog.cc:263] Verifying existing consensus state
I20260810 07:17:23.774818 31916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3: Bootstrap starting.
I20260810 07:17:23.777873 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 31836
I20260810 07:17:23.778254 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.65:41865
--local_ip_for_outbound_sockets=127.15.217.65
--tserver_master_addrs=127.15.217.126:43519
--webserver_port=46295
--webserver_interface=127.15.217.65
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:38625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:17:23.785276 31916 log.cc:824] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:23.787336 31916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:17:23.787693 31916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3: Bootstrap complete.
I20260810 07:17:23.790184 31916 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b017740ffd4355814fc558cbe98de3" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 43519 } }
I20260810 07:17:23.790441 31916 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5b017740ffd4355814fc558cbe98de3, State: Initialized, Role: FOLLOWER
I20260810 07:17:23.790650 31916 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b017740ffd4355814fc558cbe98de3" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 43519 } }
I20260810 07:17:23.790745 31916 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:17:23.790804 31916 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:17:23.790853 31916 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:17:23.791890 31916 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b017740ffd4355814fc558cbe98de3" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 43519 } }
I20260810 07:17:23.792023 31916 leader_election.cc:302] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5b017740ffd4355814fc558cbe98de3; no voters:
I20260810 07:17:23.792234 31916 leader_election.cc:288] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 07:17:23.792273 31922 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 07:17:23.792387 31922 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [term 2 LEADER]: Becoming Leader. State: Replica: b5b017740ffd4355814fc558cbe98de3, State: Running, Role: LEADER
I20260810 07:17:23.792560 31916 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:17:23.792531 31922 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b017740ffd4355814fc558cbe98de3" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 43519 } }
I20260810 07:17:23.792920 31923 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b5b017740ffd4355814fc558cbe98de3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b017740ffd4355814fc558cbe98de3" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 43519 } } }
I20260810 07:17:23.792928 31924 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5b017740ffd4355814fc558cbe98de3. Latest consensus state: current_term: 2 leader_uuid: "b5b017740ffd4355814fc558cbe98de3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b017740ffd4355814fc558cbe98de3" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 43519 } } }
I20260810 07:17:23.793149 31924 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [sys.catalog]: This master's current role is: LEADER
I20260810 07:17:23.793092 31923 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [sys.catalog]: This master's current role is: LEADER
I20260810 07:17:23.793375 31930 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:17:23.794222 31930 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:17:23.794332 31930 catalog_manager.cc:1259] Loaded cluster ID: 187ad450aac64b6492ee6c7a2add93c1
I20260810 07:17:23.794369 31930 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:17:23.794625 31930 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260810 07:17:23.799331 31930 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260810 07:17:23.801880 31930 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:17:23.802151 31930 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3: Loaded TSK: 0
I20260810 07:17:23.802431 31930 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 07:17:23.867290 31920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:23.867527 31920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:23.867570 31920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:23.868798 31920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:23.868860 31920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:23.868908 31920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.65
I20260810 07:17:23.870498 31920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:38625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.65:41865
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.217.65
--webserver_port=46295
--tserver_master_addrs=127.15.217.126:43519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.65
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:17:23.870775 31920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:23.871028 31920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:17:23.876370 31920 server_base.cc:1034] running on GCE node
W20260810 07:17:23.876296 31948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:23.876296 31946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:23.876554 31951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:23.876780 31920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:23.877053 31920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:23.878232 31920 hybrid_clock.cc:648] HybridClock initialized: now 1786346243878209 us; error 40 us; skew 500 ppm
Aug 10 07:17:23 dist-test-slave-10pc krb5kdc[31230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346243, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:23.881071 31920 init.cc:377] Logged in from keytab as kudu/127.15.217.65@KRBTEST.COM (short username kudu)
I20260810 07:17:23.881641 31920 webserver.cc:460] Webserver started at http://127.15.217.65:46295/ using document root <none> and password file <none>
I20260810 07:17:23.881933 31920 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:23.882019 31920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:23.883795 31920 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:17:23.884629 31958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:23.884788 31920 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 07:17:23.884896 31920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/wal
uuid: "d5f62e4899314ee5a13cff40ee0258a8"
format_stamp: "Formatted at 2026-08-10 07:17:23 on dist-test-slave-10pc"
server_key: "0172fd146aa36ef87b83957404b14952"
server_key_iv: "0f8c7fa7f6aba3d58ea8748dec8043e0"
server_key_version: "encryptionkey@0"
I20260810 07:17:23.885279 31920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:23.909240 31920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:23.911758 31920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:23.911947 31920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:23.912621 31920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:17:23.913136 31965 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 07:17:23.915334 31920 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 07:17:23.915405 31920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260810 07:17:23.915454 31920 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 07:17:23.916155 31920 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 07:17:23.916214 31920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260810 07:17:23.916292 31965 tablet_bootstrap.cc:492] T a55a1bce75a34d0c8cf23bc4e798832b P d5f62e4899314ee5a13cff40ee0258a8: Bootstrap starting.
I20260810 07:17:23.924026 31920 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.65:41865
I20260810 07:17:23.924073 32089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.65:41865 every 8 connection(s)
I20260810 07:17:23.924588 31920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
I20260810 07:17:23.925223 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 31920
I20260810 07:17:23.925566 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.66:44833
--local_ip_for_outbound_sockets=127.15.217.66
--tserver_master_addrs=127.15.217.126:43519
--webserver_port=37491
--webserver_interface=127.15.217.66
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:38625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:17:23.928424 31965 log.cc:824] T a55a1bce75a34d0c8cf23bc4e798832b P d5f62e4899314ee5a13cff40ee0258a8: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:23.931775 31965 tablet_bootstrap.cc:492] T a55a1bce75a34d0c8cf23bc4e798832b P d5f62e4899314ee5a13cff40ee0258a8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:17:23.932099 31965 tablet_bootstrap.cc:492] T a55a1bce75a34d0c8cf23bc4e798832b P d5f62e4899314ee5a13cff40ee0258a8: Bootstrap complete.
I20260810 07:17:23.933439 31965 ts_tablet_manager.cc:1397] T a55a1bce75a34d0c8cf23bc4e798832b P d5f62e4899314ee5a13cff40ee0258a8: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.010s
I20260810 07:17:23.934993 31965 raft_consensus.cc:348] T a55a1bce75a34d0c8cf23bc4e798832b P d5f62e4899314ee5a13cff40ee0258a8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44833 } } peers { permanent_uuid: "a8e2d66b94934297b6e280649a56d5df" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 33891 } } peers { permanent_uuid: "d5f62e4899314ee5a13cff40ee0258a8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41865 } }
I20260810 07:17:23.935321 31965 raft_consensus.cc:729] T a55a1bce75a34d0c8cf23bc4e798832b P d5f62e4899314ee5a13cff40ee0258a8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5f62e4899314ee5a13cff40ee0258a8, State: Initialized, Role: FOLLOWER
I20260810 07:17:23.935485 31965 consensus_queue.cc:260] T a55a1bce75a34d0c8cf23bc4e798832b P d5f62e4899314ee5a13cff40ee0258a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44833 } } peers { permanent_uuid: "a8e2d66b94934297b6e280649a56d5df" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 33891 } } peers { permanent_uuid: "d5f62e4899314ee5a13cff40ee0258a8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41865 } }
I20260810 07:17:23.935793 31965 ts_tablet_manager.cc:1428] T a55a1bce75a34d0c8cf23bc4e798832b P d5f62e4899314ee5a13cff40ee0258a8: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
Aug 10 07:17:23 dist-test-slave-10pc krb5kdc[31230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346243, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:17:23.943595 32093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:23.925543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:53977 (local address 127.15.217.126:43519)
0810 07:17:23.928729 (+ 3186us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:23.928742 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:23.931216 (+ 2474us) server_negotiation.cc:407] Connection header received
0810 07:17:23.932367 (+ 1151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:23.932377 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:23.932478 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:23.932608 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:23.933751 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.935196 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.936831 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.937239 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.937327 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:23.939923 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:23.939930 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:23.939930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:23.939947 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:23.941683 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:23.942143 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:23.942147 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:23.942147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:23.942251 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:23.942594 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:23.942595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:23.942595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:23.942842 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:23.942941 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:23.943246 (+ 305us) server_negotiation.cc:299] Negotiation successful
0810 07:17:23.943380 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3108,"thread_start_us":118,"threads_started":1}
I20260810 07:17:23.943620 32092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:23.925434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.65:53977)
0810 07:17:23.930961 (+ 5527us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:23.931033 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:23.932064 (+ 1031us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:23.932853 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:23.932858 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:23.933285 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:23.933612 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:23.933620 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.935317 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.935318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:23.936741 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:23.936745 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.937346 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.937347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:23.937506 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:23.938107 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:23.938119 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:23.939805 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:23.941793 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:23.941801 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:23.941804 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:23.942044 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:23.942326 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:23.942326 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:23.942327 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:23.942483 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:23.942918 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:23.942925 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:23.943067 (+ 142us) client_negotiation.cc:769] Sending connection context
0810 07:17:23.943186 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 07:17:23.943380 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5470,"thread_start_us":106,"threads_started":1}
I20260810 07:17:23.944355 32090 heartbeater.cc:347] Connected to a master server at 127.15.217.126:43519
I20260810 07:17:23.944520 32090 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:23.944753 32090 heartbeater.cc:511] Master 127.15.217.126:43519 requested a full tablet report, sending...
I20260810 07:17:23.945557 31872 ts_manager.cc:194] Registered new tserver with Master: d5f62e4899314ee5a13cff40ee0258a8 (127.15.217.65:41865)
I20260810 07:17:23.946815 31872 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.65@KRBTEST.COM'} at 127.15.217.65:53977
I20260810 07:17:23.947844 32090 heartbeater.cc:503] Master 127.15.217.126:43519 was elected leader, sending a full tablet report...
Aug 10 07:17:23 dist-test-slave-10pc krb5kdc[31230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346243, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for kudu/127.15.217.65@KRBTEST.COM
I20260810 07:17:23.955440 32099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:23.946360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60494 (local address 127.15.217.65:41865)
0810 07:17:23.946548 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:23.946553 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:23.946710 (+ 157us) server_negotiation.cc:407] Connection header received
0810 07:17:23.946795 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:23.946797 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:23.946812 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:23.946989 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:23.947417 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.948422 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.949624 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.949933 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.949990 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:23.952320 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:23.952328 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:23.952329 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:23.952336 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:23.954066 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:23.954507 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:23.954510 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:23.954510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:23.954575 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:23.954793 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:23.954794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:23.954794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:23.954950 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:23.955046 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:23.955244 (+ 198us) server_negotiation.cc:299] Negotiation successful
0810 07:17:23.955341 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":152,"thread_start_us":115,"threads_started":1}
I20260810 07:17:23.955444 32097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:23.946272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:41865 (local address 127.0.0.1:60494)
0810 07:17:23.946587 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:23.946632 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:23.946715 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:23.946907 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:23.946912 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:23.947269 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:23.947332 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:23.947334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.948588 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.948589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:23.949535 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:23.949538 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:23.950019 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:23.950020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:23.950102 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:23.950637 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:23.950644 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:23.952224 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:23.954185 (+ 1961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:23.954192 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:23.954194 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:23.954408 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:23.954644 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:23.954645 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:23.954645 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:23.954711 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:23.955023 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:23.955031 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:23.955087 (+ 56us) client_negotiation.cc:769] Sending connection context
0810 07:17:23.955194 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 07:17:23.955334 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":111,"threads_started":1}
I20260810 07:17:23.956118 32012 tablet_service.cc:1514] Processing DeleteTablet for tablet a55a1bce75a34d0c8cf23bc4e798832b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:17:23 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:60494
I20260810 07:17:23.956915 32101 tablet_replica.cc:330] stopping tablet replica
I20260810 07:17:23.957080 32101 raft_consensus.cc:2229] T a55a1bce75a34d0c8cf23bc4e798832b P d5f62e4899314ee5a13cff40ee0258a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:17:23.957158 32101 raft_consensus.cc:2258] T a55a1bce75a34d0c8cf23bc4e798832b P d5f62e4899314ee5a13cff40ee0258a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:17:23.957499 32101 ts_tablet_manager.cc:1905] T a55a1bce75a34d0c8cf23bc4e798832b P d5f62e4899314ee5a13cff40ee0258a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:17:23.960053 32101 ts_tablet_manager.cc:1918] T a55a1bce75a34d0c8cf23bc4e798832b P d5f62e4899314ee5a13cff40ee0258a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:17:23.960136 32101 log.cc:1196] T a55a1bce75a34d0c8cf23bc4e798832b P d5f62e4899314ee5a13cff40ee0258a8: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/wal/wals/a55a1bce75a34d0c8cf23bc4e798832b
I20260810 07:17:23.960397 32101 ts_tablet_manager.cc:1939] T a55a1bce75a34d0c8cf23bc4e798832b P d5f62e4899314ee5a13cff40ee0258a8: Deleting consensus metadata
I20260810 07:17:23.960968 31856 catalog_manager.cc:4841] TS d5f62e4899314ee5a13cff40ee0258a8 (127.15.217.65:41865): tablet a55a1bce75a34d0c8cf23bc4e798832b (table test-table [id=e10211a606dc4f54ba4c7bfcd57a1682]) successfully deleted
W20260810 07:17:24.018946 32094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:24.019158 32094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:24.019181 32094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:24.020370 32094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:24.020412 32094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:24.020467 32094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.66
I20260810 07:17:24.022030 32094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:38625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.66:44833
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.217.66
--webserver_port=37491
--tserver_master_addrs=127.15.217.126:43519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.66
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:17:24.022290 32094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:24.022542 32094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:24.027731 32106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:24.027797 32109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:24.027779 32094 server_base.cc:1034] running on GCE node
W20260810 07:17:24.027760 32107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:24.028271 32094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:24.028568 32094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:24.029749 32094 hybrid_clock.cc:648] HybridClock initialized: now 1786346244029731 us; error 49 us; skew 500 ppm
Aug 10 07:17:24 dist-test-slave-10pc krb5kdc[31230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346244, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:24.032390 32094 init.cc:377] Logged in from keytab as kudu/127.15.217.66@KRBTEST.COM (short username kudu)
I20260810 07:17:24.032850 32094 webserver.cc:460] Webserver started at http://127.15.217.66:37491/ using document root <none> and password file <none>
I20260810 07:17:24.033177 32094 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:24.033274 32094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:24.034770 32094 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:17:24.035405 32117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:24.035580 32094 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:24.035686 32094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/wal
uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e"
format_stamp: "Formatted at 2026-08-10 07:17:23 on dist-test-slave-10pc"
server_key: "80b71bc5edab3db9076d626e42dce357"
server_key_iv: "0aae04404f6609f798dd288d95a80dd6"
server_key_version: "encryptionkey@0"
I20260810 07:17:24.036082 32094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:24.049528 32094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:24.051548 32094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:24.051718 32094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:24.051959 32094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:17:24.052282 32094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:17:24.052337 32094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:24.052381 32094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:17:24.052405 32094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:24.058480 32094 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.66:44833
I20260810 07:17:24.058542 32243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.66:44833 every 8 connection(s)
I20260810 07:17:24.058882 32094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
I20260810 07:17:24.062844 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 32094
I20260810 07:17:24.063185 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.67:33891
--local_ip_for_outbound_sockets=127.15.217.67
--tserver_master_addrs=127.15.217.126:43519
--webserver_port=42763
--webserver_interface=127.15.217.67
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:38625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 07:17:24 dist-test-slave-10pc krb5kdc[31230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346244, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:17:24.069913 32093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:24.059688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:41759 (local address 127.15.217.126:43519)
0810 07:17:24.059761 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:24.059763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:24.059984 (+ 221us) server_negotiation.cc:407] Connection header received
0810 07:17:24.060515 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:24.060516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:24.060532 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:24.060595 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:24.061539 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:24.062051 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:24.063382 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:24.063716 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:24.063765 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:24.066687 (+ 2922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:24.066694 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:24.066694 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:24.066701 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:24.068207 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:24.068721 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:24.068722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:24.068722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:24.068783 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:24.069053 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:24.069054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:24.069054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:24.069181 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:24.069257 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:24.069644 (+ 387us) server_negotiation.cc:299] Negotiation successful
0810 07:17:24.069771 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:17:24.069939 32247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:24.059669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.66:41759)
0810 07:17:24.059843 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:24.059882 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:24.060380 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:24.060655 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:24.060658 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:24.061246 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:24.061452 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:24.061457 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:24.062142 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:24.062142 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:24.063263 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:24.063266 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:24.063868 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:24.063869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:24.064040 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:24.064663 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:24.064675 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:24.066570 (+ 1895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:24.068292 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:24.068300 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:24.068303 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:24.068625 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:24.068854 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:24.068855 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:24.068856 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:24.068980 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:24.069261 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:24.069270 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:24.069441 (+ 171us) client_negotiation.cc:769] Sending connection context
0810 07:17:24.069568 (+ 127us) client_negotiation.cc:240] Negotiation successful
0810 07:17:24.069762 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":104,"threads_started":1}
I20260810 07:17:24.070570 32245 heartbeater.cc:347] Connected to a master server at 127.15.217.126:43519
I20260810 07:17:24.070675 32245 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:24.070895 32245 heartbeater.cc:511] Master 127.15.217.126:43519 requested a full tablet report, sending...
I20260810 07:17:24.071373 31872 ts_manager.cc:194] Registered new tserver with Master: cab9a5eac3834eb6a7cfee1dbfcdad6e (127.15.217.66:44833)
I20260810 07:17:24.071771 31872 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.66@KRBTEST.COM'} at 127.15.217.66:41759
W20260810 07:17:24.151517 32249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:24.151803 32249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:24.151844 32249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:24.153151 32249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:24.153220 32249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:24.153261 32249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.67
I20260810 07:17:24.154753 32249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:38625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.67:33891
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.217.67
--webserver_port=42763
--tserver_master_addrs=127.15.217.126:43519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.67
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:17:24.155040 32249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:24.155300 32249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:24.160619 32255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:24.160665 32256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:24.160822 32249 server_base.cc:1034] running on GCE node
W20260810 07:17:24.160619 32260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:24.161136 32249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:24.161409 32249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:24.162580 32249 hybrid_clock.cc:648] HybridClock initialized: now 1786346244162559 us; error 50 us; skew 500 ppm
Aug 10 07:17:24 dist-test-slave-10pc krb5kdc[31230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346244, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:24.165372 32249 init.cc:377] Logged in from keytab as kudu/127.15.217.67@KRBTEST.COM (short username kudu)
I20260810 07:17:24.165894 32249 webserver.cc:460] Webserver started at http://127.15.217.67:42763/ using document root <none> and password file <none>
I20260810 07:17:24.166154 32249 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:24.166251 32249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:24.167735 32249 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:17:24.168431 32267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:24.168678 32249 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:17:24.168783 32249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/wal
uuid: "a8e2d66b94934297b6e280649a56d5df"
format_stamp: "Formatted at 2026-08-10 07:17:23 on dist-test-slave-10pc"
server_key: "53dd8717f0ca9e0bcd731240e692671e"
server_key_iv: "4c9a876b543b9205f1fddf19f014b87d"
server_key_version: "encryptionkey@0"
I20260810 07:17:24.169154 32249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:24.186364 32249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:24.189190 32249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:24.189368 32249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:24.189649 32249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:17:24.190052 32249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:17:24.190114 32249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:24.190158 32249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:17:24.190188 32249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:24.196358 32249 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.67:33891
I20260810 07:17:24.196434 32393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.67:33891 every 8 connection(s)
I20260810 07:17:24.196938 32249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
I20260810 07:17:24.200201 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 32249
Aug 10 07:17:24 dist-test-slave-10pc krb5kdc[31230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346244, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:17:24.208074 32399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:24.200687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39910 (local address 127.15.217.126:43519)
0810 07:17:24.200783 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:24.200784 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:24.200821 (+ 37us) server_negotiation.cc:407] Connection header received
0810 07:17:24.200905 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:24.200905 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:24.200915 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:24.200972 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:24.201705 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:24.202173 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:24.203128 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:24.203425 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:24.203500 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:24.205042 (+ 1542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:24.205044 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:24.205044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:24.205052 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:24.206667 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:24.207106 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:24.207107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:24.207107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:24.207211 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:24.207422 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:24.207423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:24.207424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:24.207589 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:24.207738 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:24.207847 (+ 109us) server_negotiation.cc:299] Negotiation successful
0810 07:17:24.207936 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76,"thread_start_us":53,"threads_started":1}
I20260810 07:17:24.208935 32093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:24.197821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:53321 (local address 127.15.217.126:43519)
0810 07:17:24.197870 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:24.197872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:24.198104 (+ 232us) server_negotiation.cc:407] Connection header received
0810 07:17:24.198667 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:24.198669 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:24.198682 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:24.198751 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:24.199768 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:24.200383 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:24.202007 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:24.202262 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:24.202332 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:24.205255 (+ 2923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:24.205257 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:24.205257 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:24.205264 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:24.207083 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:24.207652 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:24.207658 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:24.207658 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:24.207728 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:24.208081 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:24.208082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:24.208082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:24.208263 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:24.208325 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:24.208757 (+ 432us) server_negotiation.cc:299] Negotiation successful
0810 07:17:24.208845 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 07:17:24.209164 32396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:24.197789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.67:53321)
0810 07:17:24.197968 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:24.198014 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:24.198532 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:24.198845 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:24.198849 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:24.199437 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:24.199659 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:24.199664 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:24.200505 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:24.200506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:24.201913 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:24.201916 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:24.202349 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:24.202350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:24.202475 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:24.203074 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:24.203104 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:24.205147 (+ 2043us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:24.207177 (+ 2030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:24.207186 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:24.207189 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:24.207527 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:24.207817 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:24.207818 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:24.207819 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:24.207963 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:24.208355 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:24.208362 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:24.208631 (+ 269us) client_negotiation.cc:769] Sending connection context
0810 07:17:24.208734 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 07:17:24.208893 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":97,"threads_started":1}
I20260810 07:17:24.209734 32394 heartbeater.cc:347] Connected to a master server at 127.15.217.126:43519
I20260810 07:17:24.209837 32394 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:24.209990 32394 heartbeater.cc:511] Master 127.15.217.126:43519 requested a full tablet report, sending...
I20260810 07:17:24.210345 31872 ts_manager.cc:194] Registered new tserver with Master: a8e2d66b94934297b6e280649a56d5df (127.15.217.67:33891)
I20260810 07:17:24.210676 31872 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.67@KRBTEST.COM'} at 127.15.217.67:53321
I20260810 07:17:24.219887 16229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:17:24.223081 32093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:24.220274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39918 (local address 127.15.217.126:43519)
0810 07:17:24.220340 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:24.220343 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:24.220491 (+ 148us) server_negotiation.cc:407] Connection header received
0810 07:17:24.220622 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:24.220623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:24.220682 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:24.220763 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:17:24.220918 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:24.221426 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:24.222242 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:24.222459 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:24.222538 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:24.222733 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:17:24.222849 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:17:24.222944 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:24.222964 (+ 20us) server_negotiation.cc:299] Negotiation successful
0810 07:17:24.222986 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 07:17:24.223392 31872 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:39918:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:17:24.225123 31872 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:17:24.227280 32012 tablet_service.cc:1467] Processing CreateTablet for tablet 48a22f0ae2784ebe804496d9337717f0 (DEFAULT_TABLE table=test-table [id=564ec5a3f54e4dc392474f92c3988f4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:17:24.227520 32012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48a22f0ae2784ebe804496d9337717f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:17:24.229287 31965 tablet_bootstrap.cc:492] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8: Bootstrap starting.
I20260810 07:17:24.229732 31965 tablet_bootstrap.cc:654] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8: Neither blocks nor log segments found. Creating new log.
I20260810 07:17:24.230739 31965 tablet_bootstrap.cc:492] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8: No bootstrap required, opened a new log
I20260810 07:17:24.230804 31965 ts_tablet_manager.cc:1397] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 07:17:24.230964 31965 raft_consensus.cc:348] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44833 } } peers { permanent_uuid: "a8e2d66b94934297b6e280649a56d5df" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 33891 } } peers { permanent_uuid: "d5f62e4899314ee5a13cff40ee0258a8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41865 } }
I20260810 07:17:24.231062 31965 raft_consensus.cc:374] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:17:24.231117 31965 raft_consensus.cc:729] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5f62e4899314ee5a13cff40ee0258a8, State: Initialized, Role: FOLLOWER
I20260810 07:17:24.231163 31965 consensus_queue.cc:260] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44833 } } peers { permanent_uuid: "a8e2d66b94934297b6e280649a56d5df" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 33891 } } peers { permanent_uuid: "d5f62e4899314ee5a13cff40ee0258a8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41865 } }
I20260810 07:17:24.231272 31965 ts_tablet_manager.cc:1428] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:24.231536 32097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:24.227007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:44833 (local address 127.0.0.1:34674)
0810 07:17:24.227043 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:24.227055 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:24.227091 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:24.227612 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:24.227613 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:24.227615 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:17:24.227665 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:24.227667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:24.228862 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:24.228863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:24.230155 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:24.230157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:24.231223 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:24.231225 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:24.231310 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:24.231338 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 07:17:24.231405 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 07:17:24.231457 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18,"mutex_wait_us":46}
I20260810 07:17:24.231592 32402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:24.227222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34674 (local address 127.15.217.66:44833)
0810 07:17:24.227441 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:24.227446 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:24.227457 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:17:24.227500 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:24.227502 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:24.227521 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:24.227605 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:17:24.227829 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:24.228756 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:24.230258 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:24.230996 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:24.231073 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:24.231139 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:24.231424 (+ 285us) server_negotiation.cc:299] Negotiation successful
0810 07:17:24.231462 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":97,"threads_started":1}
I20260810 07:17:24.231886 32401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:24.227104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:33891 (local address 127.0.0.1:49228)
0810 07:17:24.227289 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:24.227302 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:24.227357 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:24.227716 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:24.227717 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:24.227719 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:17:24.227753 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:24.227754 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:24.228974 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:24.228974 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:24.230154 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:24.230157 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:24.231634 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:24.231635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:24.231717 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:24.231721 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:17:24.231776 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 07:17:24.231815 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":5,"thread_start_us":106,"threads_started":1}
I20260810 07:17:24.231951 32403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:24.227326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49228 (local address 127.15.217.67:33891)
0810 07:17:24.227518 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:24.227522 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:24.227533 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:17:24.227577 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:24.227579 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:24.227604 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:24.227716 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:17:24.227798 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:24.228863 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:24.230258 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:24.230996 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:24.231096 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:24.231147 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:24.231815 (+ 668us) server_negotiation.cc:299] Negotiation successful
0810 07:17:24.231866 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":102,"threads_started":1}
I20260810 07:17:24.232640 32169 tablet_service.cc:1467] Processing CreateTablet for tablet 48a22f0ae2784ebe804496d9337717f0 (DEFAULT_TABLE table=test-table [id=564ec5a3f54e4dc392474f92c3988f4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:17:24.232843 32319 tablet_service.cc:1467] Processing CreateTablet for tablet 48a22f0ae2784ebe804496d9337717f0 (DEFAULT_TABLE table=test-table [id=564ec5a3f54e4dc392474f92c3988f4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:17:24.232991 32169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48a22f0ae2784ebe804496d9337717f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:17:24.233189 32319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48a22f0ae2784ebe804496d9337717f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:17:24.236100 32405 tablet_bootstrap.cc:492] T 48a22f0ae2784ebe804496d9337717f0 P cab9a5eac3834eb6a7cfee1dbfcdad6e: Bootstrap starting.
I20260810 07:17:24.236356 32407 tablet_bootstrap.cc:492] T 48a22f0ae2784ebe804496d9337717f0 P a8e2d66b94934297b6e280649a56d5df: Bootstrap starting.
I20260810 07:17:24.236760 32405 tablet_bootstrap.cc:654] T 48a22f0ae2784ebe804496d9337717f0 P cab9a5eac3834eb6a7cfee1dbfcdad6e: Neither blocks nor log segments found. Creating new log.
I20260810 07:17:24.236999 32407 tablet_bootstrap.cc:654] T 48a22f0ae2784ebe804496d9337717f0 P a8e2d66b94934297b6e280649a56d5df: Neither blocks nor log segments found. Creating new log.
I20260810 07:17:24.237357 32405 log.cc:824] T 48a22f0ae2784ebe804496d9337717f0 P cab9a5eac3834eb6a7cfee1dbfcdad6e: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:24.237726 32407 log.cc:824] T 48a22f0ae2784ebe804496d9337717f0 P a8e2d66b94934297b6e280649a56d5df: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:24.238215 32405 tablet_bootstrap.cc:492] T 48a22f0ae2784ebe804496d9337717f0 P cab9a5eac3834eb6a7cfee1dbfcdad6e: No bootstrap required, opened a new log
I20260810 07:17:24.238261 32405 ts_tablet_manager.cc:1397] T 48a22f0ae2784ebe804496d9337717f0 P cab9a5eac3834eb6a7cfee1dbfcdad6e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:17:24.239768 32405 raft_consensus.cc:348] T 48a22f0ae2784ebe804496d9337717f0 P cab9a5eac3834eb6a7cfee1dbfcdad6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44833 } } peers { permanent_uuid: "a8e2d66b94934297b6e280649a56d5df" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 33891 } } peers { permanent_uuid: "d5f62e4899314ee5a13cff40ee0258a8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41865 } }
I20260810 07:17:24.239888 32405 raft_consensus.cc:374] T 48a22f0ae2784ebe804496d9337717f0 P cab9a5eac3834eb6a7cfee1dbfcdad6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:17:24.239908 32405 raft_consensus.cc:729] T 48a22f0ae2784ebe804496d9337717f0 P cab9a5eac3834eb6a7cfee1dbfcdad6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cab9a5eac3834eb6a7cfee1dbfcdad6e, State: Initialized, Role: FOLLOWER
I20260810 07:17:24.240032 32405 consensus_queue.cc:260] T 48a22f0ae2784ebe804496d9337717f0 P cab9a5eac3834eb6a7cfee1dbfcdad6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44833 } } peers { permanent_uuid: "a8e2d66b94934297b6e280649a56d5df" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 33891 } } peers { permanent_uuid: "d5f62e4899314ee5a13cff40ee0258a8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41865 } }
I20260810 07:17:24.240315 32405 ts_tablet_manager.cc:1428] T 48a22f0ae2784ebe804496d9337717f0 P cab9a5eac3834eb6a7cfee1dbfcdad6e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:17:24.240355 32245 heartbeater.cc:503] Master 127.15.217.126:43519 was elected leader, sending a full tablet report...
I20260810 07:17:24.241539 32407 tablet_bootstrap.cc:492] T 48a22f0ae2784ebe804496d9337717f0 P a8e2d66b94934297b6e280649a56d5df: No bootstrap required, opened a new log
I20260810 07:17:24.241681 32407 ts_tablet_manager.cc:1397] T 48a22f0ae2784ebe804496d9337717f0 P a8e2d66b94934297b6e280649a56d5df: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260810 07:17:24.243750 32407 raft_consensus.cc:348] T 48a22f0ae2784ebe804496d9337717f0 P a8e2d66b94934297b6e280649a56d5df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44833 } } peers { permanent_uuid: "a8e2d66b94934297b6e280649a56d5df" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 33891 } } peers { permanent_uuid: "d5f62e4899314ee5a13cff40ee0258a8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41865 } }
I20260810 07:17:24.243937 32407 raft_consensus.cc:374] T 48a22f0ae2784ebe804496d9337717f0 P a8e2d66b94934297b6e280649a56d5df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:17:24.243975 32407 raft_consensus.cc:729] T 48a22f0ae2784ebe804496d9337717f0 P a8e2d66b94934297b6e280649a56d5df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8e2d66b94934297b6e280649a56d5df, State: Initialized, Role: FOLLOWER
I20260810 07:17:24.244108 32407 consensus_queue.cc:260] T 48a22f0ae2784ebe804496d9337717f0 P a8e2d66b94934297b6e280649a56d5df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44833 } } peers { permanent_uuid: "a8e2d66b94934297b6e280649a56d5df" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 33891 } } peers { permanent_uuid: "d5f62e4899314ee5a13cff40ee0258a8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41865 } }
I20260810 07:17:24.244431 32407 ts_tablet_manager.cc:1428] T 48a22f0ae2784ebe804496d9337717f0 P a8e2d66b94934297b6e280649a56d5df: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 07:17:24.244609 32394 heartbeater.cc:503] Master 127.15.217.126:43519 was elected leader, sending a full tablet report...
I20260810 07:17:24.308441 32411 raft_consensus.cc:482] T 48a22f0ae2784ebe804496d9337717f0 P cab9a5eac3834eb6a7cfee1dbfcdad6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:17:24.308609 32411 raft_consensus.cc:504] T 48a22f0ae2784ebe804496d9337717f0 P cab9a5eac3834eb6a7cfee1dbfcdad6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44833 } } peers { permanent_uuid: "a8e2d66b94934297b6e280649a56d5df" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 33891 } } peers { permanent_uuid: "d5f62e4899314ee5a13cff40ee0258a8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41865 } }
I20260810 07:17:24.309001 32411 leader_election.cc:288] T 48a22f0ae2784ebe804496d9337717f0 P cab9a5eac3834eb6a7cfee1dbfcdad6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8e2d66b94934297b6e280649a56d5df (127.15.217.67:33891), d5f62e4899314ee5a13cff40ee0258a8 (127.15.217.65:41865)
W20260810 07:17:24.309507 32246 tablet.cc:2369] T 48a22f0ae2784ebe804496d9337717f0 P cab9a5eac3834eb6a7cfee1dbfcdad6e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:17:24.312994 32247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:24.309194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:33891 (local address 127.15.217.66:50899)
0810 07:17:24.309253 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:24.309266 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:24.309331 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:24.309567 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:24.309568 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:24.309570 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:17:24.309669 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:24.309672 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:24.310602 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:24.310603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:24.311851 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:24.311854 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:24.312670 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:24.312671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:24.312781 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:24.312786 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:17:24.312849 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 07:17:24.312907 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":1}
I20260810 07:17:24.312997 32403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:24.309282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:50899 (local address 127.15.217.67:33891)
0810 07:17:24.309329 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:24.309331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:24.309341 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:17:24.309432 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:24.309434 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:24.309448 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:24.309508 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:17:24.309759 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:24.310497 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:24.311958 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:24.312579 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:24.312670 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:24.312681 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:24.312877 (+ 196us) server_negotiation.cc:299] Negotiation successful
0810 07:17:24.312930 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 07:17:24.313020 32099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:24.309307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:42425 (local address 127.15.217.65:41865)
0810 07:17:24.309371 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:24.309373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:24.309521 (+ 148us) server_negotiation.cc:407] Connection header received
0810 07:17:24.309558 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:24.309560 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:24.309591 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:24.309661 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:17:24.309900 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:24.310651 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:24.312025 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:24.312691 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:24.312760 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:24.312846 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:24.312916 (+ 70us) server_negotiation.cc:299] Negotiation successful
0810 07:17:24.312957 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 07:17:24.313041 32413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:24.309238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:41865 (local address 127.15.217.66:42425)
0810 07:17:24.309420 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:24.309431 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:24.309496 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:24.309715 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:24.309716 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:24.309718 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:17:24.309800 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:24.309802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:24.310751 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:24.310752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:24.311855 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:24.311857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:24.312784 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:24.312785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:24.312860 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:24.312863 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:17:24.312913 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 07:17:24.312944 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":3,"thread_start_us":100,"threads_started":1}
I20260810 07:17:24.313405 32342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48a22f0ae2784ebe804496d9337717f0" candidate_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8e2d66b94934297b6e280649a56d5df" is_pre_election: true
I20260810 07:17:24.313370 32037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48a22f0ae2784ebe804496d9337717f0" candidate_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5f62e4899314ee5a13cff40ee0258a8" is_pre_election: true
I20260810 07:17:24.313541 32037 raft_consensus.cc:2454] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cab9a5eac3834eb6a7cfee1dbfcdad6e in term 0.
I20260810 07:17:24.313601 32342 raft_consensus.cc:2454] T 48a22f0ae2784ebe804496d9337717f0 P a8e2d66b94934297b6e280649a56d5df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cab9a5eac3834eb6a7cfee1dbfcdad6e in term 0.
I20260810 07:17:24.313827 32119 leader_election.cc:302] T 48a22f0ae2784ebe804496d9337717f0 P cab9a5eac3834eb6a7cfee1dbfcdad6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cab9a5eac3834eb6a7cfee1dbfcdad6e, d5f62e4899314ee5a13cff40ee0258a8; no voters:
I20260810 07:17:24.313999 32411 raft_consensus.cc:2781] T 48a22f0ae2784ebe804496d9337717f0 P cab9a5eac3834eb6a7cfee1dbfcdad6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:17:24.314064 32411 raft_consensus.cc:482] T 48a22f0ae2784ebe804496d9337717f0 P cab9a5eac3834eb6a7cfee1dbfcdad6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:17:24.314082 32411 raft_consensus.cc:3037] T 48a22f0ae2784ebe804496d9337717f0 P cab9a5eac3834eb6a7cfee1dbfcdad6e [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:17:24.314807 32411 raft_consensus.cc:504] T 48a22f0ae2784ebe804496d9337717f0 P cab9a5eac3834eb6a7cfee1dbfcdad6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44833 } } peers { permanent_uuid: "a8e2d66b94934297b6e280649a56d5df" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 33891 } } peers { permanent_uuid: "d5f62e4899314ee5a13cff40ee0258a8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41865 } }
I20260810 07:17:24.314963 32411 leader_election.cc:288] T 48a22f0ae2784ebe804496d9337717f0 P cab9a5eac3834eb6a7cfee1dbfcdad6e [CANDIDATE]: Term 1 election: Requested vote from peers a8e2d66b94934297b6e280649a56d5df (127.15.217.67:33891), d5f62e4899314ee5a13cff40ee0258a8 (127.15.217.65:41865)
I20260810 07:17:24.315119 32342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48a22f0ae2784ebe804496d9337717f0" candidate_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8e2d66b94934297b6e280649a56d5df"
I20260810 07:17:24.315119 32037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48a22f0ae2784ebe804496d9337717f0" candidate_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5f62e4899314ee5a13cff40ee0258a8"
I20260810 07:17:24.315205 32037 raft_consensus.cc:3037] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:17:24.315207 32342 raft_consensus.cc:3037] T 48a22f0ae2784ebe804496d9337717f0 P a8e2d66b94934297b6e280649a56d5df [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:17:24.316140 32037 raft_consensus.cc:2454] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cab9a5eac3834eb6a7cfee1dbfcdad6e in term 1.
I20260810 07:17:24.316140 32342 raft_consensus.cc:2454] T 48a22f0ae2784ebe804496d9337717f0 P a8e2d66b94934297b6e280649a56d5df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cab9a5eac3834eb6a7cfee1dbfcdad6e in term 1.
I20260810 07:17:24.316370 32118 leader_election.cc:302] T 48a22f0ae2784ebe804496d9337717f0 P cab9a5eac3834eb6a7cfee1dbfcdad6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8e2d66b94934297b6e280649a56d5df, cab9a5eac3834eb6a7cfee1dbfcdad6e; no voters:
I20260810 07:17:24.316524 32411 raft_consensus.cc:2781] T 48a22f0ae2784ebe804496d9337717f0 P cab9a5eac3834eb6a7cfee1dbfcdad6e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:17:24.316666 32411 raft_consensus.cc:686] T 48a22f0ae2784ebe804496d9337717f0 P cab9a5eac3834eb6a7cfee1dbfcdad6e [term 1 LEADER]: Becoming Leader. State: Replica: cab9a5eac3834eb6a7cfee1dbfcdad6e, State: Running, Role: LEADER
I20260810 07:17:24.316778 32411 consensus_queue.cc:237] T 48a22f0ae2784ebe804496d9337717f0 P cab9a5eac3834eb6a7cfee1dbfcdad6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44833 } } peers { permanent_uuid: "a8e2d66b94934297b6e280649a56d5df" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 33891 } } peers { permanent_uuid: "d5f62e4899314ee5a13cff40ee0258a8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41865 } }
I20260810 07:17:24.317703 31872 catalog_manager.cc:5495] T 48a22f0ae2784ebe804496d9337717f0 P cab9a5eac3834eb6a7cfee1dbfcdad6e reported cstate change: term changed from 0 to 1, leader changed from <none> to cab9a5eac3834eb6a7cfee1dbfcdad6e (127.15.217.66). New cstate: current_term: 1 leader_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44833 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8e2d66b94934297b6e280649a56d5df" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 33891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5f62e4899314ee5a13cff40ee0258a8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41865 } health_report { overall_health: UNKNOWN } } }
I20260810 07:17:24.324070 32402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:24.321855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34676 (local address 127.15.217.66:44833)
0810 07:17:24.321913 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:24.321919 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:24.321926 (+ 7us) server_negotiation.cc:407] Connection header received
0810 07:17:24.321959 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:24.321959 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:24.321975 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:24.322036 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:17:24.322181 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:24.322654 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:24.323344 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:24.323527 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:24.323575 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:24.323709 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:17:24.323808 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:17:24.323871 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:24.323972 (+ 101us) server_negotiation.cc:299] Negotiation successful
0810 07:17:24.323992 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 07:17:24.326009 32342 raft_consensus.cc:1261] T 48a22f0ae2784ebe804496d9337717f0 P a8e2d66b94934297b6e280649a56d5df [term 1 FOLLOWER]: Refusing update from remote peer cab9a5eac3834eb6a7cfee1dbfcdad6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:17:24.326009 32037 raft_consensus.cc:1261] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [term 1 FOLLOWER]: Refusing update from remote peer cab9a5eac3834eb6a7cfee1dbfcdad6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:17:24.326378 32414 consensus_queue.cc:1035] T 48a22f0ae2784ebe804496d9337717f0 P cab9a5eac3834eb6a7cfee1dbfcdad6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8e2d66b94934297b6e280649a56d5df" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 33891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:17:24.326520 32411 consensus_queue.cc:1035] T 48a22f0ae2784ebe804496d9337717f0 P cab9a5eac3834eb6a7cfee1dbfcdad6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5f62e4899314ee5a13cff40ee0258a8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:17:24.329372 32419 mvcc.cc:205] Tried to move back new op lower bound from 7316874216756314112 to 7316874216722329600. Current Snapshot: MvccSnapshot[applied={T|T < 7316874216756314112 or (T in {7316874216756314112})}]
I20260810 07:17:24.330065 32418 mvcc.cc:205] Tried to move back new op lower bound from 7316874216756314112 to 7316874216722329600. Current Snapshot: MvccSnapshot[applied={T|T < 7316874216756314112}]
I20260810 07:17:24.331305 31871 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:39918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:17:24.331403 31871 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:39918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:17:24.332150 31871 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3: Sending DeleteTablet for 3 replicas of tablet 48a22f0ae2784ebe804496d9337717f0
I20260810 07:17:24.332526 32012 tablet_service.cc:1514] Processing DeleteTablet for tablet 48a22f0ae2784ebe804496d9337717f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:17:24 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:60494
I20260810 07:17:24.332540 32169 tablet_service.cc:1514] Processing DeleteTablet for tablet 48a22f0ae2784ebe804496d9337717f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:17:24 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:34674
I20260810 07:17:24.332554 32319 tablet_service.cc:1514] Processing DeleteTablet for tablet 48a22f0ae2784ebe804496d9337717f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:17:24 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:49228
I20260810 07:17:24.332629 32101 tablet_replica.cc:330] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8: stopping tablet replica
I20260810 07:17:24.332684 32101 raft_consensus.cc:2229] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:17:24.332751 32101 raft_consensus.cc:2258] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:17:24.332967 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 31920
I20260810 07:17:24.332986 32428 tablet_replica.cc:330] T 48a22f0ae2784ebe804496d9337717f0 P cab9a5eac3834eb6a7cfee1dbfcdad6e: stopping tablet replica
I20260810 07:17:24.332984 32426 tablet_replica.cc:330] T 48a22f0ae2784ebe804496d9337717f0 P a8e2d66b94934297b6e280649a56d5df: stopping tablet replica
I20260810 07:17:24.333194 32426 raft_consensus.cc:2229] T 48a22f0ae2784ebe804496d9337717f0 P a8e2d66b94934297b6e280649a56d5df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:17:24.333349 32426 raft_consensus.cc:2258] T 48a22f0ae2784ebe804496d9337717f0 P a8e2d66b94934297b6e280649a56d5df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:17:24.334820 32428 raft_consensus.cc:2229] T 48a22f0ae2784ebe804496d9337717f0 P cab9a5eac3834eb6a7cfee1dbfcdad6e [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:17:24.334982 32428 raft_consensus.cc:2258] T 48a22f0ae2784ebe804496d9337717f0 P cab9a5eac3834eb6a7cfee1dbfcdad6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:17:24.335110 32426 ts_tablet_manager.cc:1905] T 48a22f0ae2784ebe804496d9337717f0 P a8e2d66b94934297b6e280649a56d5df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:17:24.335749 32428 ts_tablet_manager.cc:1905] T 48a22f0ae2784ebe804496d9337717f0 P cab9a5eac3834eb6a7cfee1dbfcdad6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:17:24.337177 32426 ts_tablet_manager.cc:1918] T 48a22f0ae2784ebe804496d9337717f0 P a8e2d66b94934297b6e280649a56d5df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:17:24.337334 32426 log.cc:1196] T 48a22f0ae2784ebe804496d9337717f0 P a8e2d66b94934297b6e280649a56d5df: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/wal/wals/48a22f0ae2784ebe804496d9337717f0
I20260810 07:17:24.337743 32426 ts_tablet_manager.cc:1939] T 48a22f0ae2784ebe804496d9337717f0 P a8e2d66b94934297b6e280649a56d5df: Deleting consensus metadata
I20260810 07:17:24.338271 32428 ts_tablet_manager.cc:1918] T 48a22f0ae2784ebe804496d9337717f0 P cab9a5eac3834eb6a7cfee1dbfcdad6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:17:24.338376 32428 log.cc:1196] T 48a22f0ae2784ebe804496d9337717f0 P cab9a5eac3834eb6a7cfee1dbfcdad6e: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/wal/wals/48a22f0ae2784ebe804496d9337717f0
I20260810 07:17:24.338356 31855 catalog_manager.cc:4841] TS a8e2d66b94934297b6e280649a56d5df (127.15.217.67:33891): tablet 48a22f0ae2784ebe804496d9337717f0 (table test-table [id=564ec5a3f54e4dc392474f92c3988f4c]) successfully deleted
I20260810 07:17:24.338799 32428 ts_tablet_manager.cc:1939] T 48a22f0ae2784ebe804496d9337717f0 P cab9a5eac3834eb6a7cfee1dbfcdad6e: Deleting consensus metadata
I20260810 07:17:24.339433 31855 catalog_manager.cc:4841] TS cab9a5eac3834eb6a7cfee1dbfcdad6e (127.15.217.66:44833): tablet 48a22f0ae2784ebe804496d9337717f0 (table test-table [id=564ec5a3f54e4dc392474f92c3988f4c]) successfully deleted
W20260810 07:17:24.348148 31856 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.217.65:41865 (error 108)
I20260810 07:17:24.348376 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 32094
I20260810 07:17:24.348827 32401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:24.348694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:41865 (local address 127.0.0.1:60508)
0810 07:17:24.348749 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:24.348784 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.65:41865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 07:17:24.348950 31856 catalog_manager.cc:4584] TS d5f62e4899314ee5a13cff40ee0258a8 (127.15.217.65:41865): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 48a22f0ae2784ebe804496d9337717f0: Network error: Client connection negotiation failed: client connection to 127.15.217.65:41865: connect: Connection refused (error 111)
I20260810 07:17:24.356428 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 32249
I20260810 07:17:24.363202 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 31836
I20260810 07:17:24.371335 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:43519
--webserver_interface=127.15.217.126
--webserver_port=43441
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:38625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:43519
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:17:24.457815 32430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:24.458033 32430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:24.458065 32430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:24.459172 32430 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:17:24.459205 32430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:24.459219 32430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:24.459231 32430 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:17:24.459244 32430 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:17:24.460667 32430 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:38625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:43519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:43519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=43441
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:17:24.460871 32430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:24.461069 32430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:24.466171 32436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:24.466146 32438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:24.466173 32435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:24.466526 32430 server_base.cc:1034] running on GCE node
I20260810 07:17:24.466811 32430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:24.467135 32430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:24.468281 32430 hybrid_clock.cc:648] HybridClock initialized: now 1786346244468259 us; error 39 us; skew 500 ppm
Aug 10 07:17:24 dist-test-slave-10pc krb5kdc[31230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346244, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:24.470960 32430 init.cc:377] Logged in from keytab as kudu/127.15.217.126@KRBTEST.COM (short username kudu)
I20260810 07:17:24.471390 32430 webserver.cc:460] Webserver started at http://127.15.217.126:43441/ using document root <none> and password file <none>
I20260810 07:17:24.471582 32430 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:24.471629 32430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:24.473191 32430 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:17:24.473903 32445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:24.474174 32430 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:17:24.474288 32430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "b5b017740ffd4355814fc558cbe98de3"
format_stamp: "Formatted at 2026-08-10 07:17:22 on dist-test-slave-10pc"
server_key: "9cf669d801a5ea9d14511819eaf4ab2b"
server_key_iv: "959cdcfc8ac39d4c29d7d9f1a271b1c8"
server_key_version: "encryptionkey@0"
I20260810 07:17:24.474697 32430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:24.484942 32430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:24.487217 32430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:24.487407 32430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:24.491638 32430 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:43519
I20260810 07:17:24.491677 32503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:43519 every 8 connection(s)
I20260810 07:17:24.492110 32430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:17:24.493824 32504 sys_catalog.cc:263] Verifying existing consensus state
I20260810 07:17:24.494531 32504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3: Bootstrap starting.
I20260810 07:17:24.497454 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 32430
I20260810 07:17:24.497789 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.65:41865
--local_ip_for_outbound_sockets=127.15.217.65
--tserver_master_addrs=127.15.217.126:43519
--webserver_port=46295
--webserver_interface=127.15.217.65
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:38625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:17:24.504789 32504 log.cc:824] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:24.507220 32504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:17:24.507601 32504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3: Bootstrap complete.
I20260810 07:17:24.510176 32504 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b017740ffd4355814fc558cbe98de3" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 43519 } }
I20260810 07:17:24.510434 32504 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5b017740ffd4355814fc558cbe98de3, State: Initialized, Role: FOLLOWER
I20260810 07:17:24.510622 32504 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b017740ffd4355814fc558cbe98de3" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 43519 } }
I20260810 07:17:24.510703 32504 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:17:24.510754 32504 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:17:24.510805 32504 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [term 2 FOLLOWER]: Advancing to term 3
I20260810 07:17:24.511931 32504 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b017740ffd4355814fc558cbe98de3" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 43519 } }
I20260810 07:17:24.512059 32504 leader_election.cc:302] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5b017740ffd4355814fc558cbe98de3; no voters:
I20260810 07:17:24.512277 32504 leader_election.cc:288] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 07:17:24.512331 32508 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 07:17:24.512497 32508 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [term 3 LEADER]: Becoming Leader. State: Replica: b5b017740ffd4355814fc558cbe98de3, State: Running, Role: LEADER
I20260810 07:17:24.512609 32504 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:17:24.512786 32508 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b017740ffd4355814fc558cbe98de3" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 43519 } }
I20260810 07:17:24.513085 32508 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "b5b017740ffd4355814fc558cbe98de3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b017740ffd4355814fc558cbe98de3" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 43519 } } }
I20260810 07:17:24.513185 32508 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [sys.catalog]: This master's current role is: LEADER
I20260810 07:17:24.513149 32509 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5b017740ffd4355814fc558cbe98de3. Latest consensus state: current_term: 3 leader_uuid: "b5b017740ffd4355814fc558cbe98de3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b017740ffd4355814fc558cbe98de3" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 43519 } } }
I20260810 07:17:24.513582 32509 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5b017740ffd4355814fc558cbe98de3 [sys.catalog]: This master's current role is: LEADER
I20260810 07:17:24.514000 32517 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:17:24.514918 32517 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:17:24.515070 32517 catalog_manager.cc:1259] Loaded cluster ID: 187ad450aac64b6492ee6c7a2add93c1
I20260810 07:17:24.515126 32517 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260810 07:17:24.515403 32517 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fc8785be9ad google::LogMessage::Fail() at ??:0
@ 0x7fc8785c188a google::LogMessage::SendToLog() at ??:0
@ 0x7fc8785be4de google::LogMessage::Flush() at ??:0
@ 0x7fc8785bfa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fc8804df0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fc8804e88b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fc87a48c0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fc87a481954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fc87b2e56db start_thread at ??:0
@ 0x7fc8770bc61f clone at ??:0
W20260810 07:17:24.605420 32506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:24.605651 32506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:24.605680 32506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:24.606905 32506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:24.606959 32506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:24.606990 32506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.65
I20260810 07:17:24.608680 32506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:38625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.65:41865
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.217.65
--webserver_port=46295
--tserver_master_addrs=127.15.217.126:43519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.65
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:17:24.608979 32506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:24.609249 32506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:24.614849 32545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:24.614768 32541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:24.615043 32506 server_base.cc:1034] running on GCE node
W20260810 07:17:24.614784 32543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:24.615484 32506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:24.615777 32506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:24.616925 32506 hybrid_clock.cc:648] HybridClock initialized: now 1786346244616910 us; error 31 us; skew 500 ppm
Aug 10 07:17:24 dist-test-slave-10pc krb5kdc[31230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346244, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:24.619542 32506 init.cc:377] Logged in from keytab as kudu/127.15.217.65@KRBTEST.COM (short username kudu)
I20260810 07:17:24.620054 32506 webserver.cc:460] Webserver started at http://127.15.217.65:46295/ using document root <none> and password file <none>
I20260810 07:17:24.620296 32506 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:24.620343 32506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:24.621830 32506 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:17:24.622613 32552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:24.622824 32506 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:17:24.622946 32506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/wal
uuid: "d5f62e4899314ee5a13cff40ee0258a8"
format_stamp: "Formatted at 2026-08-10 07:17:23 on dist-test-slave-10pc"
server_key: "0172fd146aa36ef87b83957404b14952"
server_key_iv: "0f8c7fa7f6aba3d58ea8748dec8043e0"
server_key_version: "encryptionkey@0"
I20260810 07:17:24.623351 32506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:24.637862 32506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:24.640339 32506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:24.640564 32506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:24.640856 32506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:17:24.641397 32561 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 07:17:24.643482 32506 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 07:17:24.643559 32506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260810 07:17:24.643607 32506 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 07:17:24.644415 32506 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 07:17:24.644510 32506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.001s
I20260810 07:17:24.644588 32561 tablet_bootstrap.cc:492] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8: Bootstrap starting.
I20260810 07:17:24.652199 32506 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.65:41865
I20260810 07:17:24.652370 32678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.65:41865 every 8 connection(s)
I20260810 07:17:24.652691 32506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
I20260810 07:17:24.655383 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 32506
I20260810 07:17:24.655714 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.66:44833
--local_ip_for_outbound_sockets=127.15.217.66
--tserver_master_addrs=127.15.217.126:43519
--webserver_port=37491
--webserver_interface=127.15.217.66
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:38625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:17:24.656522 32561 log.cc:824] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:24.659440 32561 tablet_bootstrap.cc:492] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:17:24.659788 32561 tablet_bootstrap.cc:492] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8: Bootstrap complete.
I20260810 07:17:24.661237 32561 ts_tablet_manager.cc:1397] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.014s
I20260810 07:17:24.662695 32561 raft_consensus.cc:348] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44833 } } peers { permanent_uuid: "a8e2d66b94934297b6e280649a56d5df" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 33891 } } peers { permanent_uuid: "d5f62e4899314ee5a13cff40ee0258a8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41865 } }
I20260810 07:17:24.663051 32561 raft_consensus.cc:729] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5f62e4899314ee5a13cff40ee0258a8, State: Initialized, Role: FOLLOWER
I20260810 07:17:24.663240 32561 consensus_queue.cc:260] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44833 } } peers { permanent_uuid: "a8e2d66b94934297b6e280649a56d5df" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 33891 } } peers { permanent_uuid: "d5f62e4899314ee5a13cff40ee0258a8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41865 } }
I20260810 07:17:24.663537 32561 ts_tablet_manager.cc:1428] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
W20260810 07:17:24.716316 32681 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:17:24.653433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.65:58345)
0810 07:17:24.653615 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:24.653663 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:24.654187 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:24.716132 (+ 61945us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":108,"threads_started":1}
W20260810 07:17:24.716619 32679 heartbeater.cc:644] Failed to heartbeat to 127.15.217.126:43519 (0 consecutive failures): Network error: Failed to ping master at 127.15.217.126:43519: Client connection negotiation failed: client connection to 127.15.217.126:43519: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 07:17:24.716988 32681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:24.716858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.65:33949)
0810 07:17:24.716917 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:24.716967 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 07:17:24.717319 32681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:24.717221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.65:49401)
0810 07:17:24.717269 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:24.717293 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 07:17:24.717425 32679 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 07:17:24.749895 32683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:24.750154 32683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:24.750197 32683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:24.751384 32683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:24.751451 32683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:24.751498 32683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.66
I20260810 07:17:24.753221 32683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:38625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.66:44833
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.217.66
--webserver_port=37491
--tserver_master_addrs=127.15.217.126:43519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.66
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:17:24.753509 32683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:24.753768 32683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:24.759030 32691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:24.759021 32693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:24.759024 32690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:24.759050 32683 server_base.cc:1034] running on GCE node
I20260810 07:17:24.759476 32683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:24.759742 32683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:24.760918 32683 hybrid_clock.cc:648] HybridClock initialized: now 1786346244760906 us; error 40 us; skew 500 ppm
Aug 10 07:17:24 dist-test-slave-10pc krb5kdc[31230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346244, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:24.763664 32683 init.cc:377] Logged in from keytab as kudu/127.15.217.66@KRBTEST.COM (short username kudu)
I20260810 07:17:24.764072 32683 webserver.cc:460] Webserver started at http://127.15.217.66:37491/ using document root <none> and password file <none>
I20260810 07:17:24.764284 32683 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:24.764328 32683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:24.765764 32683 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:17:24.766362 32701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:24.766574 32683 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:17:24.766667 32683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/wal
uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e"
format_stamp: "Formatted at 2026-08-10 07:17:23 on dist-test-slave-10pc"
server_key: "80b71bc5edab3db9076d626e42dce357"
server_key_iv: "0aae04404f6609f798dd288d95a80dd6"
server_key_version: "encryptionkey@0"
I20260810 07:17:24.767000 32683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:24.785188 32683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:24.787360 32683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:24.787483 32683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:24.787746 32683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:17:24.788066 32683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:17:24.788100 32683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:24.788122 32683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:17:24.788136 32683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:24.794451 32683 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.66:44833
I20260810 07:17:24.794510 352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.66:44833 every 8 connection(s)
I20260810 07:17:24.794883 32683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
I20260810 07:17:24.796101 355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:24.795619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.66:55931)
0810 07:17:24.795847 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:24.795938 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":82,"threads_started":1}
W20260810 07:17:24.796653 353 heartbeater.cc:644] Failed to heartbeat to 127.15.217.126:43519 (0 consecutive failures): Network error: Failed to ping master at 127.15.217.126:43519: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
I20260810 07:17:24.796990 355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:24.796852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.66:34469)
0810 07:17:24.796899 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:24.796929 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260810 07:17:24.797274 355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:24.797211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.66:48047)
0810 07:17:24.797237 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:24.797259 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260810 07:17:24.797401 353 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 07:17:24.803685 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 32683
I20260810 07:17:24.804000 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.67:33891
--local_ip_for_outbound_sockets=127.15.217.67
--tserver_master_addrs=127.15.217.126:43519
--webserver_port=42763
--webserver_interface=127.15.217.67
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:38625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:17:24.890664 356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:24.890885 356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:24.890910 356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:24.892117 356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:24.892164 356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:24.892192 356 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.67
I20260810 07:17:24.893787 356 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:38625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.67:33891
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.217.67
--webserver_port=42763
--tserver_master_addrs=127.15.217.126:43519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.67
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:17:24.894068 356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:24.894330 356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:24.899574 366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:24.899600 364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:24.899650 362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:24.899914 356 server_base.cc:1034] running on GCE node
I20260810 07:17:24.900269 356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:24.900722 356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:24.901947 356 hybrid_clock.cc:648] HybridClock initialized: now 1786346244901925 us; error 37 us; skew 500 ppm
Aug 10 07:17:24 dist-test-slave-10pc krb5kdc[31230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346244, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:24.905058 356 init.cc:377] Logged in from keytab as kudu/127.15.217.67@KRBTEST.COM (short username kudu)
I20260810 07:17:24.905613 356 webserver.cc:460] Webserver started at http://127.15.217.67:42763/ using document root <none> and password file <none>
I20260810 07:17:24.905886 356 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:24.905967 356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:24.907536 356 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:17:24.908161 373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:24.908401 356 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:17:24.908526 356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/wal
uuid: "a8e2d66b94934297b6e280649a56d5df"
format_stamp: "Formatted at 2026-08-10 07:17:23 on dist-test-slave-10pc"
server_key: "53dd8717f0ca9e0bcd731240e692671e"
server_key_iv: "4c9a876b543b9205f1fddf19f014b87d"
server_key_version: "encryptionkey@0"
I20260810 07:17:24.908908 356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:24.927129 356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:24.929706 356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:24.929883 356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:24.930193 356 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:17:24.930552 356 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:17:24.930613 356 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:24.930651 356 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:17:24.930678 356 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:24.937273 356 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.67:33891
I20260810 07:17:24.937405 502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.67:33891 every 8 connection(s)
I20260810 07:17:24.937754 356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
I20260810 07:17:24.938891 505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:24.938495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.67:33739)
0810 07:17:24.938664 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:24.938748 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":79,"threads_started":1}
W20260810 07:17:24.939321 503 heartbeater.cc:644] Failed to heartbeat to 127.15.217.126:43519 (0 consecutive failures): Network error: Failed to ping master at 127.15.217.126:43519: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
I20260810 07:17:24.939666 505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:24.939515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.67:45711)
0810 07:17:24.939602 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:24.939641 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260810 07:17:24.940034 505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:24.939898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.67:33979)
0810 07:17:24.939942 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:24.939975 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260810 07:17:24.940192 503 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 07:17:24.940431 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 356
I20260810 07:17:25.718080 507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:25.717813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.65:41393)
0810 07:17:25.717966 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:25.718010 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":70,"threads_started":1}
I20260810 07:17:25.797971 508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:25.797755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.66:34635)
0810 07:17:25.797878 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:25.797918 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":62,"threads_started":1}
I20260810 07:17:25.940815 509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:25.940596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.67:59061)
0810 07:17:25.940719 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:25.940756 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":62,"threads_started":1}
I20260810 07:17:25.969513 510 raft_consensus.cc:482] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:17:25.969702 510 raft_consensus.cc:504] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44833 } } peers { permanent_uuid: "a8e2d66b94934297b6e280649a56d5df" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 33891 } } peers { permanent_uuid: "d5f62e4899314ee5a13cff40ee0258a8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41865 } }
I20260810 07:17:25.970110 510 leader_election.cc:288] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cab9a5eac3834eb6a7cfee1dbfcdad6e (127.15.217.66:44833), a8e2d66b94934297b6e280649a56d5df (127.15.217.67:33891)
Aug 10 07:17:25 dist-test-slave-10pc krb5kdc[31230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346244, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for kudu/127.15.217.66@KRBTEST.COM
Aug 10 07:17:25 dist-test-slave-10pc krb5kdc[31230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346244, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for kudu/127.15.217.67@KRBTEST.COM
I20260810 07:17:25.981019 511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:25.970396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:48289 (local address 127.15.217.66:44833)
0810 07:17:25.970626 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:25.970641 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:25.970687 (+ 46us) server_negotiation.cc:407] Connection header received
0810 07:17:25.971171 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:25.971174 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:25.971212 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:25.971368 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:25.972170 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:25.973152 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:25.974194 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:25.974591 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:25.974708 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:25.977105 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:25.977109 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:25.977109 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:25.977121 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:25.979189 (+ 2068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:25.979731 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:25.979734 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:25.979735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:25.979829 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:25.980085 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:25.980086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:25.980086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:25.980342 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:25.980432 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:25.980769 (+ 337us) server_negotiation.cc:299] Negotiation successful
0810 07:17:25.980894 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":74,"threads_started":1}
I20260810 07:17:25.981003 507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:25.970291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:44833 (local address 127.15.217.65:48289)
0810 07:17:25.970346 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:25.970357 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:25.970417 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:25.971398 (+ 981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:25.971401 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:25.971840 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:25.972026 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:25.972033 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:25.973302 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:25.973303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:25.974090 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:25.974093 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:25.974683 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:25.974684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:25.974798 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:25.975415 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:25.975434 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:25.976982 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:25.979302 (+ 2320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:25.979310 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:25.979312 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:25.979642 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:25.979904 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:25.979905 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:25.979905 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:25.980017 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:25.980433 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:25.980435 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:25.980594 (+ 159us) client_negotiation.cc:769] Sending connection context
0810 07:17:25.980709 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 07:17:25.980904 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":56}
I20260810 07:17:25.981034 513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:25.970557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:55205 (local address 127.15.217.67:33891)
0810 07:17:25.970688 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:25.970697 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:25.970731 (+ 34us) server_negotiation.cc:407] Connection header received
0810 07:17:25.971161 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:25.971163 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:25.971204 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:25.971377 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:25.972170 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:25.973126 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:25.974187 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:25.974590 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:25.974720 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:25.977522 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:25.977526 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:25.977526 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:25.977536 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:25.979246 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:25.979730 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:25.979733 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:25.979734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:25.979827 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:25.980088 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:25.980089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:25.980090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:25.980341 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:25.980430 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:25.980768 (+ 338us) server_negotiation.cc:299] Negotiation successful
0810 07:17:25.980896 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":52,"threads_started":1}
I20260810 07:17:25.981132 512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:25.970417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:33891 (local address 127.15.217.65:55205)
0810 07:17:25.970516 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:25.970526 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:25.970584 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:25.971399 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:25.971403 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:25.971840 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:25.972026 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:25.972033 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:25.973251 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:25.973253 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:25.974090 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:25.974092 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:25.974702 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:25.974703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:25.974799 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:25.975475 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:25.975480 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:25.977405 (+ 1925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:25.979342 (+ 1937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:25.979343 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:25.979343 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:25.979643 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:25.979897 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:25.979898 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:25.979898 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:25.980013 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:25.980424 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:25.980431 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:25.980593 (+ 162us) client_negotiation.cc:769] Sending connection context
0810 07:17:25.980709 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 07:17:25.980988 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":194,"thread_start_us":61,"threads_started":1}
W20260810 07:17:25.981984 32553 leader_election.cc:341] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a8e2d66b94934297b6e280649a56d5df (127.15.217.67:33891): Not found: Tablet not found: 48a22f0ae2784ebe804496d9337717f0
W20260810 07:17:25.982173 32553 leader_election.cc:341] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cab9a5eac3834eb6a7cfee1dbfcdad6e (127.15.217.66:44833): Not found: Tablet not found: 48a22f0ae2784ebe804496d9337717f0
I20260810 07:17:25.982232 32553 leader_election.cc:302] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d5f62e4899314ee5a13cff40ee0258a8; no voters: a8e2d66b94934297b6e280649a56d5df, cab9a5eac3834eb6a7cfee1dbfcdad6e
I20260810 07:17:25.982347 510 raft_consensus.cc:2726] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:17:26.718968 514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:26.718660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.65:45481)
0810 07:17:26.718829 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:26.718896 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":77,"threads_started":1}
I20260810 07:17:26.798687 515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:26.798473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.66:47705)
0810 07:17:26.798607 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:26.798646 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":74,"threads_started":1}
I20260810 07:17:26.941557 516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:26.941316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.67:35663)
0810 07:17:26.941472 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:26.941511 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":93,"threads_started":1}
I20260810 07:17:27.719781 517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:27.719529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.65:55629)
0810 07:17:27.719684 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:27.719731 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":66,"threads_started":1}
I20260810 07:17:27.724112 518 raft_consensus.cc:482] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:17:27.724251 518 raft_consensus.cc:504] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44833 } } peers { permanent_uuid: "a8e2d66b94934297b6e280649a56d5df" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 33891 } } peers { permanent_uuid: "d5f62e4899314ee5a13cff40ee0258a8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41865 } }
I20260810 07:17:27.724439 518 leader_election.cc:288] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cab9a5eac3834eb6a7cfee1dbfcdad6e (127.15.217.66:44833), a8e2d66b94934297b6e280649a56d5df (127.15.217.67:33891)
W20260810 07:17:27.724977 32553 leader_election.cc:341] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cab9a5eac3834eb6a7cfee1dbfcdad6e (127.15.217.66:44833): Not found: Tablet not found: 48a22f0ae2784ebe804496d9337717f0
W20260810 07:17:27.725062 32553 leader_election.cc:341] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a8e2d66b94934297b6e280649a56d5df (127.15.217.67:33891): Not found: Tablet not found: 48a22f0ae2784ebe804496d9337717f0
I20260810 07:17:27.725082 32553 leader_election.cc:302] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d5f62e4899314ee5a13cff40ee0258a8; no voters: a8e2d66b94934297b6e280649a56d5df, cab9a5eac3834eb6a7cfee1dbfcdad6e
I20260810 07:17:27.725158 518 raft_consensus.cc:2726] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:17:27.799374 519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:27.799149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.66:43149)
0810 07:17:27.799294 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:27.799329 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":68,"threads_started":1}
I20260810 07:17:27.942344 522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:27.942075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.67:60757)
0810 07:17:27.942250 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:27.942295 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":89,"threads_started":1}
I20260810 07:17:28.720956 523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:28.720408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.65:41559)
0810 07:17:28.720797 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:28.720865 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":137,"threads_started":1}
I20260810 07:17:28.800201 524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:28.799901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.66:39617)
0810 07:17:28.800111 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:28.800155 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":114,"threads_started":1}
I20260810 07:17:28.943387 525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:28.943022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.67:45753)
0810 07:17:28.943267 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:28.943318 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":125,"threads_started":1}
I20260810 07:17:29.368856 526 raft_consensus.cc:482] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:17:29.368974 526 raft_consensus.cc:504] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44833 } } peers { permanent_uuid: "a8e2d66b94934297b6e280649a56d5df" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 33891 } } peers { permanent_uuid: "d5f62e4899314ee5a13cff40ee0258a8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41865 } }
I20260810 07:17:29.369216 526 leader_election.cc:288] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cab9a5eac3834eb6a7cfee1dbfcdad6e (127.15.217.66:44833), a8e2d66b94934297b6e280649a56d5df (127.15.217.67:33891)
W20260810 07:17:29.369724 32553 leader_election.cc:341] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a8e2d66b94934297b6e280649a56d5df (127.15.217.67:33891): Not found: Tablet not found: 48a22f0ae2784ebe804496d9337717f0
W20260810 07:17:29.369817 32553 leader_election.cc:341] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cab9a5eac3834eb6a7cfee1dbfcdad6e (127.15.217.66:44833): Not found: Tablet not found: 48a22f0ae2784ebe804496d9337717f0
I20260810 07:17:29.369837 32553 leader_election.cc:302] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d5f62e4899314ee5a13cff40ee0258a8; no voters: a8e2d66b94934297b6e280649a56d5df, cab9a5eac3834eb6a7cfee1dbfcdad6e
I20260810 07:17:29.369930 526 raft_consensus.cc:2726] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:17:29.721871 527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:29.721576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.65:37335)
0810 07:17:29.721735 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:29.721781 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":92,"threads_started":1}
I20260810 07:17:29.801088 528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:29.800836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.66:60775)
0810 07:17:29.800993 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:29.801041 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":84,"threads_started":1}
I20260810 07:17:29.944170 529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:29.943924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.67:42435)
0810 07:17:29.944081 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:29.944122 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":81,"threads_started":1}
I20260810 07:17:30.722653 530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:30.722377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.65:60547)
0810 07:17:30.722534 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:30.722603 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":67,"threads_started":1}
I20260810 07:17:30.801837 531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:30.801600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.66:49713)
0810 07:17:30.801746 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:30.801789 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":67,"threads_started":1}
I20260810 07:17:30.945036 532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:30.944752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.67:33529)
0810 07:17:30.944939 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:30.944980 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":85,"threads_started":1}
I20260810 07:17:31.225741 533 raft_consensus.cc:482] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:17:31.225845 533 raft_consensus.cc:504] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44833 } } peers { permanent_uuid: "a8e2d66b94934297b6e280649a56d5df" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 33891 } } peers { permanent_uuid: "d5f62e4899314ee5a13cff40ee0258a8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41865 } }
I20260810 07:17:31.226136 533 leader_election.cc:288] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cab9a5eac3834eb6a7cfee1dbfcdad6e (127.15.217.66:44833), a8e2d66b94934297b6e280649a56d5df (127.15.217.67:33891)
W20260810 07:17:31.226613 32553 leader_election.cc:341] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a8e2d66b94934297b6e280649a56d5df (127.15.217.67:33891): Not found: Tablet not found: 48a22f0ae2784ebe804496d9337717f0
W20260810 07:17:31.226720 32553 leader_election.cc:341] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cab9a5eac3834eb6a7cfee1dbfcdad6e (127.15.217.66:44833): Not found: Tablet not found: 48a22f0ae2784ebe804496d9337717f0
I20260810 07:17:31.226752 32553 leader_election.cc:302] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d5f62e4899314ee5a13cff40ee0258a8; no voters: a8e2d66b94934297b6e280649a56d5df, cab9a5eac3834eb6a7cfee1dbfcdad6e
I20260810 07:17:31.226856 533 raft_consensus.cc:2726] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:17:31.723454 534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:31.723212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.65:57261)
0810 07:17:31.723354 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:31.723403 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":87,"threads_started":1}
I20260810 07:17:31.802734 535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:31.802433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.66:42441)
0810 07:17:31.802646 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:31.802690 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":115,"threads_started":1}
I20260810 07:17:31.945894 536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:31.945613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.67:53131)
0810 07:17:31.945798 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:31.945842 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":80,"threads_started":1}
I20260810 07:17:32.724324 538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:32.724044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.65:48659)
0810 07:17:32.724209 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:32.724262 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":86,"threads_started":1}
I20260810 07:17:32.803546 539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:32.803301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.66:59301)
0810 07:17:32.803452 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:32.803496 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":83,"threads_started":1}
I20260810 07:17:32.946632 540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:32.946419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.67:53695)
0810 07:17:32.946547 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:32.946583 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":68,"threads_started":1}
I20260810 07:17:33.725399 541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:33.725104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.65:36143)
0810 07:17:33.725275 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:33.725332 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":87,"threads_started":1}
I20260810 07:17:33.804383 542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:33.804130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.66:37967)
0810 07:17:33.804282 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:33.804326 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":68,"threads_started":1}
I20260810 07:17:33.947374 543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:33.947129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.67:42367)
0810 07:17:33.947280 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:33.947324 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":69,"threads_started":1}
I20260810 07:17:34.726233 544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:34.725972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.65:57805)
0810 07:17:34.726138 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:34.726177 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":114,"threads_started":1}
I20260810 07:17:34.805219 545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:34.804967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.66:38993)
0810 07:17:34.805126 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:34.805170 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":67,"threads_started":1}
I20260810 07:17:34.948319 546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:34.948083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.67:53685)
0810 07:17:34.948234 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:34.948271 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
I20260810 07:17:35.609676 547 raft_consensus.cc:482] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:17:35.609791 547 raft_consensus.cc:504] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44833 } } peers { permanent_uuid: "a8e2d66b94934297b6e280649a56d5df" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 33891 } } peers { permanent_uuid: "d5f62e4899314ee5a13cff40ee0258a8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41865 } }
I20260810 07:17:35.609932 547 leader_election.cc:288] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cab9a5eac3834eb6a7cfee1dbfcdad6e (127.15.217.66:44833), a8e2d66b94934297b6e280649a56d5df (127.15.217.67:33891)
W20260810 07:17:35.610450 32553 leader_election.cc:341] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cab9a5eac3834eb6a7cfee1dbfcdad6e (127.15.217.66:44833): Not found: Tablet not found: 48a22f0ae2784ebe804496d9337717f0
W20260810 07:17:35.610545 32553 leader_election.cc:341] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a8e2d66b94934297b6e280649a56d5df (127.15.217.67:33891): Not found: Tablet not found: 48a22f0ae2784ebe804496d9337717f0
I20260810 07:17:35.610584 32553 leader_election.cc:302] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d5f62e4899314ee5a13cff40ee0258a8; no voters: a8e2d66b94934297b6e280649a56d5df, cab9a5eac3834eb6a7cfee1dbfcdad6e
I20260810 07:17:35.610698 547 raft_consensus.cc:2726] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:17:35.727001 549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.726768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.65:37371)
0810 07:17:35.726901 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:35.726947 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":66,"threads_started":1}
I20260810 07:17:35.805989 550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.805747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.66:51831)
0810 07:17:35.805895 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:35.805939 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":70,"threads_started":1}
I20260810 07:17:35.949178 551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.948889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.67:57789)
0810 07:17:35.949080 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:35.949127 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":106,"threads_started":1}
I20260810 07:17:36.727806 552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:36.727545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.65:40863)
0810 07:17:36.727706 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:36.727755 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":85,"threads_started":1}
I20260810 07:17:36.806874 553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:36.806611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.66:55893)
0810 07:17:36.806779 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:36.806825 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":89,"threads_started":1}
I20260810 07:17:36.949922 554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:36.949698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.67:34605)
0810 07:17:36.949832 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:36.949873 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":66,"threads_started":1}
I20260810 07:17:37.728642 555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:37.728354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.65:37009)
0810 07:17:37.728533 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:37.728586 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":75,"threads_started":1}
I20260810 07:17:37.807727 557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:37.807463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.66:60183)
0810 07:17:37.807621 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:37.807680 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":66,"threads_started":1}
I20260810 07:17:37.950809 558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:37.950526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.67:53601)
0810 07:17:37.950706 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:37.950756 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":100,"threads_started":1}
I20260810 07:17:38.729458 559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:38.729213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.65:59705)
0810 07:17:38.729351 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:38.729406 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":68,"threads_started":1}
I20260810 07:17:38.808554 560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:38.808272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.66:50559)
0810 07:17:38.808432 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:38.808504 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":70,"threads_started":1}
I20260810 07:17:38.951726 561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:38.951454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.67:53113)
0810 07:17:38.951624 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:38.951675 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":91,"threads_started":1}
I20260810 07:17:39.329878 563 raft_consensus.cc:482] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:17:39.330031 563 raft_consensus.cc:504] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cab9a5eac3834eb6a7cfee1dbfcdad6e" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 44833 } } peers { permanent_uuid: "a8e2d66b94934297b6e280649a56d5df" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 33891 } } peers { permanent_uuid: "d5f62e4899314ee5a13cff40ee0258a8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41865 } }
I20260810 07:17:39.330257 563 leader_election.cc:288] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cab9a5eac3834eb6a7cfee1dbfcdad6e (127.15.217.66:44833), a8e2d66b94934297b6e280649a56d5df (127.15.217.67:33891)
W20260810 07:17:39.330863 32553 leader_election.cc:341] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a8e2d66b94934297b6e280649a56d5df (127.15.217.67:33891): Not found: Tablet not found: 48a22f0ae2784ebe804496d9337717f0
W20260810 07:17:39.330935 32553 leader_election.cc:341] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cab9a5eac3834eb6a7cfee1dbfcdad6e (127.15.217.66:44833): Not found: Tablet not found: 48a22f0ae2784ebe804496d9337717f0
I20260810 07:17:39.330984 32553 leader_election.cc:302] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d5f62e4899314ee5a13cff40ee0258a8; no voters: a8e2d66b94934297b6e280649a56d5df, cab9a5eac3834eb6a7cfee1dbfcdad6e
I20260810 07:17:39.331079 563 raft_consensus.cc:2726] T 48a22f0ae2784ebe804496d9337717f0 P d5f62e4899314ee5a13cff40ee0258a8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:17:39.730348 564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:39.730094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.65:44469)
0810 07:17:39.730246 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:39.730292 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":89,"threads_started":1}
I20260810 07:17:39.809389 565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:39.809174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.66:54691)
0810 07:17:39.809303 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:39.809342 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":68,"threads_started":1}
I20260810 07:17:39.949169 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 32506
I20260810 07:17:39.952930 566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:39.952478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:43519 (local address 127.15.217.67:35397)
0810 07:17:39.952828 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:39.952866 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:43519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":277,"threads_started":1}
I20260810 07:17:39.958336 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 32683
I20260810 07:17:39.965101 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 356
2026-08-10T07:17:39Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18349 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:17:40 dist-test-slave-10pc krb5kdc[569](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:17:40 dist-test-slave-10pc krb5kdc[569](info): set up 2 sockets
Aug 10 07:17:40 dist-test-slave-10pc krb5kdc[569](info): commencing operation
krb5kdc: starting...
W20260810 07:17:41.065217 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:17:41 dist-test-slave-10pc krb5kdc[569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:17:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:17:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
I20260810 07:17:41.199120 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:35977
--webserver_interface=127.15.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:33041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:35977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:17:41.285804 582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:41.286211 582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:41.286285 582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:41.287671 582 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 07:17:41.287743 582 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 07:17:41.287766 582 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 07:17:41.287787 582 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:17:41.287819 582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:41.287859 582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:41.287889 582 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:17:41.287914 582 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:17:41.289619 582 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:33041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:35977
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:35977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:17:41.289934 582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:41.290203 582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:41.295662 589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:41.295754 588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:41.295810 582 server_base.cc:1034] running on GCE node
W20260810 07:17:41.295725 591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:41.296239 582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:41.296602 582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:41.297789 582 hybrid_clock.cc:648] HybridClock initialized: now 1786346261297776 us; error 30 us; skew 500 ppm
Aug 10 07:17:41 dist-test-slave-10pc krb5kdc[569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346261, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:41.305999 582 init.cc:377] Logged in from keytab as kudu/127.15.217.126@KRBTEST.COM (short username kudu)
I20260810 07:17:41.306542 582 webserver.cc:460] Webserver started at http://127.15.217.126:38241/ using document root <none> and password file <none>
I20260810 07:17:41.306774 582 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:41.306818 582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:41.306915 582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:17:41.308003 582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "1928338c4bf347c1b69f77261844dca1"
format_stamp: "Formatted at 2026-08-10 07:17:41 on dist-test-slave-10pc"
server_key: "59269af0ebc5c8a76e00fa02e1bc9eb8"
server_key_iv: "88f9a2ec389d8b71ba46eb8e610de5d2"
server_key_version: "encryptionkey@0"
I20260810 07:17:41.308328 582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "1928338c4bf347c1b69f77261844dca1"
format_stamp: "Formatted at 2026-08-10 07:17:41 on dist-test-slave-10pc"
server_key: "59269af0ebc5c8a76e00fa02e1bc9eb8"
server_key_iv: "88f9a2ec389d8b71ba46eb8e610de5d2"
server_key_version: "encryptionkey@0"
I20260810 07:17:41.309856 582 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:17:41.310693 598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:41.310891 582 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:17:41.311008 582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "1928338c4bf347c1b69f77261844dca1"
format_stamp: "Formatted at 2026-08-10 07:17:41 on dist-test-slave-10pc"
server_key: "59269af0ebc5c8a76e00fa02e1bc9eb8"
server_key_iv: "88f9a2ec389d8b71ba46eb8e610de5d2"
server_key_version: "encryptionkey@0"
I20260810 07:17:41.311091 582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:41.334653 582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:41.336850 582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:41.337054 582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:41.342012 582 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:35977
I20260810 07:17:41.342062 658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:35977 every 8 connection(s)
I20260810 07:17:41.342476 582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:17:41.343484 659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:17:41.345863 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 582
I20260810 07:17:41.345997 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:17:41.346099 659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1: Bootstrap starting.
I20260810 07:17:41.346254 16229 external_mini_cluster.cc:1428] Setting key 730cb0dac1efe28d442ad028cb96b492
I20260810 07:17:41.346918 659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1: Neither blocks nor log segments found. Creating new log.
I20260810 07:17:41.347317 659 log.cc:824] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:41.348187 659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1: No bootstrap required, opened a new log
I20260810 07:17:41.350756 659 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1928338c4bf347c1b69f77261844dca1" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35977 } }
I20260810 07:17:41.350901 659 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:17:41.350924 659 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1928338c4bf347c1b69f77261844dca1, State: Initialized, Role: FOLLOWER
I20260810 07:17:41.351074 659 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1928338c4bf347c1b69f77261844dca1" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35977 } }
I20260810 07:17:41.351155 659 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:17:41.351179 659 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:17:41.351280 659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:17:41.351892 659 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1928338c4bf347c1b69f77261844dca1" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35977 } }
I20260810 07:17:41.352028 659 leader_election.cc:302] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1928338c4bf347c1b69f77261844dca1; no voters:
I20260810 07:17:41.352239 659 leader_election.cc:288] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:17:41.352290 664 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:17:41.352553 659 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:17:41.353590 664 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [term 1 LEADER]: Becoming Leader. State: Replica: 1928338c4bf347c1b69f77261844dca1, State: Running, Role: LEADER
I20260810 07:17:41.353768 664 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1928338c4bf347c1b69f77261844dca1" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35977 } }
Aug 10 07:17:41 dist-test-slave-10pc krb5kdc[569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:17:41.354401 665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1928338c4bf347c1b69f77261844dca1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1928338c4bf347c1b69f77261844dca1" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35977 } } }
I20260810 07:17:41.354503 665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [sys.catalog]: This master's current role is: LEADER
I20260810 07:17:41.354928 673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1928338c4bf347c1b69f77261844dca1. Latest consensus state: current_term: 1 leader_uuid: "1928338c4bf347c1b69f77261844dca1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1928338c4bf347c1b69f77261844dca1" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35977 } } }
I20260810 07:17:41.354929 680 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:17:41.355038 673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [sys.catalog]: This master's current role is: LEADER
I20260810 07:17:41.355561 680 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:17:41.357561 680 catalog_manager.cc:1347] Generated new cluster ID: 721586d3450246ceabce0cda93082d71
I20260810 07:17:41.357613 680 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:17:41.358827 662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:41.346882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34560 (local address 127.15.217.126:35977)
0810 07:17:41.347168 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:41.347179 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:41.347215 (+ 36us) server_negotiation.cc:407] Connection header received
0810 07:17:41.347842 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:41.347852 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:41.347920 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:41.348081 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:41.348842 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.350292 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.351239 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.351730 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.351869 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:41.354796 (+ 2927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:41.354800 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:41.354801 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:41.354814 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:41.357243 (+ 2429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:41.357624 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:41.357628 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:41.357628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:41.357729 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:41.357950 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:41.357951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:41.357951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:41.358210 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:41.358351 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:41.358481 (+ 130us) server_negotiation.cc:299] Negotiation successful
0810 07:17:41.358642 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":96,"threads_started":1}
I20260810 07:17:41.380486 680 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:17:41.382167 680 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:17:41.387395 680 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1: Generated new TSK 0
I20260810 07:17:41.387622 680 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab.
Entry for principal kudu/127.15.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab.
I20260810 07:17:41.420145 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.65:0
--local_ip_for_outbound_sockets=127.15.217.65
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:35977
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:33041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:17:41.507997 689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:41.508241 689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:41.508282 689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:41.509675 689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:41.509753 689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:41.509792 689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.65
I20260810 07:17:41.511643 689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:33041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:35977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.65
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:17:41.511850 689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:41.512041 689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:41.517573 694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:41.517547 695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:41.517596 698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:41.517886 689 server_base.cc:1034] running on GCE node
I20260810 07:17:41.518149 689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:41.518406 689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:41.519578 689 hybrid_clock.cc:648] HybridClock initialized: now 1786346261519555 us; error 35 us; skew 500 ppm
Aug 10 07:17:41 dist-test-slave-10pc krb5kdc[569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346261, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:41.522368 689 init.cc:377] Logged in from keytab as kudu/127.15.217.65@KRBTEST.COM (short username kudu)
I20260810 07:17:41.522877 689 webserver.cc:460] Webserver started at http://127.15.217.65:36741/ using document root <none> and password file <none>
I20260810 07:17:41.523176 689 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:41.523258 689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:41.523444 689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:17:41.524765 689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/data/instance:
uuid: "7a71dca4901f4ab4b2bc099682fb3ec2"
format_stamp: "Formatted at 2026-08-10 07:17:41 on dist-test-slave-10pc"
server_key: "66148bb75b5517b2d9382b42069e2b3e"
server_key_iv: "eb0ea3b6d15a7b8715ac91d65b6a307a"
server_key_version: "encryptionkey@0"
I20260810 07:17:41.525213 689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance:
uuid: "7a71dca4901f4ab4b2bc099682fb3ec2"
format_stamp: "Formatted at 2026-08-10 07:17:41 on dist-test-slave-10pc"
server_key: "66148bb75b5517b2d9382b42069e2b3e"
server_key_iv: "eb0ea3b6d15a7b8715ac91d65b6a307a"
server_key_version: "encryptionkey@0"
I20260810 07:17:41.526804 689 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:17:41.527660 705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:41.527844 689 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:17:41.527956 689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/wal
uuid: "7a71dca4901f4ab4b2bc099682fb3ec2"
format_stamp: "Formatted at 2026-08-10 07:17:41 on dist-test-slave-10pc"
server_key: "66148bb75b5517b2d9382b42069e2b3e"
server_key_iv: "eb0ea3b6d15a7b8715ac91d65b6a307a"
server_key_version: "encryptionkey@0"
I20260810 07:17:41.528036 689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:41.559136 689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:41.561627 689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:41.561851 689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:41.562171 689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:17:41.562532 689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:17:41.562599 689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:41.562651 689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:17:41.562677 689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:41.569017 689 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.65:42967
I20260810 07:17:41.569087 831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.65:42967 every 8 connection(s)
I20260810 07:17:41.569453 689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
Aug 10 07:17:41 dist-test-slave-10pc krb5kdc[569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346261, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:17:41.577443 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 689
I20260810 07:17:41.577579 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance
I20260810 07:17:41.577821 16229 external_mini_cluster.cc:1428] Setting key 4c3ea19d717f3d98f31201682cb40114
I20260810 07:17:41.579753 662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:41.570268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:34957 (local address 127.15.217.126:35977)
0810 07:17:41.570339 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:41.570341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:41.570544 (+ 203us) server_negotiation.cc:407] Connection header received
0810 07:17:41.571053 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:41.571054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:41.571065 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:41.571130 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:41.572161 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.572788 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.573882 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.574102 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.574165 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:41.576421 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:41.576423 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:41.576423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:41.576431 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:41.578166 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:41.578701 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:41.578702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:41.578702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:41.578769 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:41.579086 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:41.579087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:41.579087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:41.579195 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:41.579247 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:41.579609 (+ 362us) server_negotiation.cc:299] Negotiation successful
0810 07:17:41.579687 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 07:17:41.580082 835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:41.570236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.65:34957)
0810 07:17:41.570402 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:41.570448 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:41.570958 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:41.571189 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:41.571192 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:41.571815 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:41.572045 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:41.572052 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.572912 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.572913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:41.573780 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:41.573783 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.574205 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.574207 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:41.574318 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:41.574834 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:41.574843 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:41.576331 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:41.578265 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:41.578275 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:41.578278 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:41.578591 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:41.578842 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:41.578843 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:41.578844 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:41.579009 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:41.579262 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:41.579270 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:41.579456 (+ 186us) client_negotiation.cc:769] Sending connection context
0810 07:17:41.579579 (+ 123us) client_negotiation.cc:240] Negotiation successful
0810 07:17:41.579786 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":87,"threads_started":1}
I20260810 07:17:41.580752 832 heartbeater.cc:347] Connected to a master server at 127.15.217.126:35977
I20260810 07:17:41.580879 832 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:41.581086 832 heartbeater.cc:511] Master 127.15.217.126:35977 requested a full tablet report, sending...
I20260810 07:17:41.581732 617 ts_manager.cc:194] Registered new tserver with Master: 7a71dca4901f4ab4b2bc099682fb3ec2 (127.15.217.65:42967)
I20260810 07:17:41.582504 617 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.65@KRBTEST.COM'} at 127.15.217.65:34957
WARNING: no policy specified for kudu/127.15.217.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab.
Entry for principal kudu/127.15.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab.
I20260810 07:17:41.607645 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.66:0
--local_ip_for_outbound_sockets=127.15.217.66
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:35977
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:33041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:17:41.693266 838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:41.693516 838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:41.693554 838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:41.694733 838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:41.694811 838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:41.694855 838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.66
I20260810 07:17:41.696426 838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:33041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:35977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.66
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:17:41.696784 838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:41.697041 838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:41.702417 843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:41.702425 846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:41.702459 838 server_base.cc:1034] running on GCE node
W20260810 07:17:41.702435 844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:41.702893 838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:41.703184 838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:41.704356 838 hybrid_clock.cc:648] HybridClock initialized: now 1786346261704352 us; error 36 us; skew 500 ppm
Aug 10 07:17:41 dist-test-slave-10pc krb5kdc[569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346261, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:41.707006 838 init.cc:377] Logged in from keytab as kudu/127.15.217.66@KRBTEST.COM (short username kudu)
I20260810 07:17:41.707464 838 webserver.cc:460] Webserver started at http://127.15.217.66:45335/ using document root <none> and password file <none>
I20260810 07:17:41.707713 838 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:41.707763 838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:41.707937 838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:17:41.709115 838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/data/instance:
uuid: "308b655a3d4d4ba793d2ac7f4e74cd92"
format_stamp: "Formatted at 2026-08-10 07:17:41 on dist-test-slave-10pc"
server_key: "0558f8690cf26aacf559a4a0f1fa565e"
server_key_iv: "cf8e179fb61258221c96cf6df0335485"
server_key_version: "encryptionkey@0"
I20260810 07:17:41.709555 838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance:
uuid: "308b655a3d4d4ba793d2ac7f4e74cd92"
format_stamp: "Formatted at 2026-08-10 07:17:41 on dist-test-slave-10pc"
server_key: "0558f8690cf26aacf559a4a0f1fa565e"
server_key_iv: "cf8e179fb61258221c96cf6df0335485"
server_key_version: "encryptionkey@0"
I20260810 07:17:41.710927 838 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:17:41.711670 854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:41.711865 838 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 07:17:41.711958 838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/wal
uuid: "308b655a3d4d4ba793d2ac7f4e74cd92"
format_stamp: "Formatted at 2026-08-10 07:17:41 on dist-test-slave-10pc"
server_key: "0558f8690cf26aacf559a4a0f1fa565e"
server_key_iv: "cf8e179fb61258221c96cf6df0335485"
server_key_version: "encryptionkey@0"
I20260810 07:17:41.712018 838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:41.732986 838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:41.735152 838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:41.735276 838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:41.735482 838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:17:41.735855 838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:17:41.735894 838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:41.735918 838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:17:41.735972 838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:41.742445 838 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.66:36221
I20260810 07:17:41.742499 975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.66:36221 every 8 connection(s)
I20260810 07:17:41.742861 838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
I20260810 07:17:41.743826 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 838
I20260810 07:17:41.743966 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance
I20260810 07:17:41.744195 16229 external_mini_cluster.cc:1428] Setting key 2f72d24326d84086df738e8adbd07c74
Aug 10 07:17:41 dist-test-slave-10pc krb5kdc[569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346261, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:17:41.753788 662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:41.743659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:37781 (local address 127.15.217.126:35977)
0810 07:17:41.743740 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:41.743743 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:41.743941 (+ 198us) server_negotiation.cc:407] Connection header received
0810 07:17:41.744644 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:41.744645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:41.744660 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:41.744723 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:41.745854 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.746649 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.747729 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.747946 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.748029 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:41.750456 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:41.750458 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:41.750458 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:41.750468 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:41.752123 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:41.752659 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:41.752661 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:41.752661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:41.752733 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:41.753002 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:41.753003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:41.753003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:41.753174 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:41.753256 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:41.753590 (+ 334us) server_negotiation.cc:299] Negotiation successful
0810 07:17:41.753690 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 07:17:41.754051 979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:41.743591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.66:37781)
0810 07:17:41.743782 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:41.743828 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:41.744492 (+ 664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:41.744784 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:41.744788 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:41.745507 (+ 719us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:41.745735 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:41.745742 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.746762 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.746762 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:41.747604 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:41.747606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.748041 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.748042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:41.748188 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:41.748791 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:41.748801 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:41.750326 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:41.752228 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:41.752238 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:41.752241 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:41.752576 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:41.752793 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:41.752794 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:41.752794 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:41.752923 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:41.753246 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:41.753256 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:41.753420 (+ 164us) client_negotiation.cc:769] Sending connection context
0810 07:17:41.753533 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 07:17:41.753740 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":78,"threads_started":1}
WARNING: no policy specified for kudu/127.15.217.67@KRBTEST.COM; defaulting to no policy
I20260810 07:17:41.755328 977 heartbeater.cc:347] Connected to a master server at 127.15.217.126:35977
I20260810 07:17:41.755455 977 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:41.755698 977 heartbeater.cc:511] Master 127.15.217.126:35977 requested a full tablet report, sending...
I20260810 07:17:41.756162 617 ts_manager.cc:194] Registered new tserver with Master: 308b655a3d4d4ba793d2ac7f4e74cd92 (127.15.217.66:36221)
I20260810 07:17:41.756714 617 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.66@KRBTEST.COM'} at 127.15.217.66:37781
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab.
Entry for principal kudu/127.15.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab.
I20260810 07:17:41.775491 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.67:0
--local_ip_for_outbound_sockets=127.15.217.67
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:35977
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:33041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:17:41.862843 982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:41.863045 982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:41.863070 982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:41.864219 982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:41.864261 982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:41.864287 982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.67
I20260810 07:17:41.865849 982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:33041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:35977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.67
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:17:41.866108 982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:41.866350 982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:41.871577 988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:41.871595 987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:41.871578 990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:41.871599 982 server_base.cc:1034] running on GCE node
I20260810 07:17:41.872012 982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:41.872274 982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:41.873443 982 hybrid_clock.cc:648] HybridClock initialized: now 1786346261873433 us; error 36 us; skew 500 ppm
Aug 10 07:17:41 dist-test-slave-10pc krb5kdc[569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346261, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:41.876037 982 init.cc:377] Logged in from keytab as kudu/127.15.217.67@KRBTEST.COM (short username kudu)
I20260810 07:17:41.876559 982 webserver.cc:460] Webserver started at http://127.15.217.67:35797/ using document root <none> and password file <none>
I20260810 07:17:41.876837 982 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:41.876915 982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:41.877074 982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:17:41.878144 982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/data/instance:
uuid: "c55b7534c4324b52a0ef9fb73002d48c"
format_stamp: "Formatted at 2026-08-10 07:17:41 on dist-test-slave-10pc"
server_key: "ef0590cbfab821d2293d13d4770e2089"
server_key_iv: "473ba2ffda0a22d91714f5a63f8a5566"
server_key_version: "encryptionkey@0"
I20260810 07:17:41.878535 982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance:
uuid: "c55b7534c4324b52a0ef9fb73002d48c"
format_stamp: "Formatted at 2026-08-10 07:17:41 on dist-test-slave-10pc"
server_key: "ef0590cbfab821d2293d13d4770e2089"
server_key_iv: "473ba2ffda0a22d91714f5a63f8a5566"
server_key_version: "encryptionkey@0"
I20260810 07:17:41.879889 982 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:17:41.880673 998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:41.880856 982 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:17:41.880962 982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/wal
uuid: "c55b7534c4324b52a0ef9fb73002d48c"
format_stamp: "Formatted at 2026-08-10 07:17:41 on dist-test-slave-10pc"
server_key: "ef0590cbfab821d2293d13d4770e2089"
server_key_iv: "473ba2ffda0a22d91714f5a63f8a5566"
server_key_version: "encryptionkey@0"
I20260810 07:17:41.881047 982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:41.892292 982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:41.894243 982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:41.894411 982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:41.894649 982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:17:41.894975 982 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:17:41.895031 982 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:41.895062 982 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:17:41.895083 982 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:41.901001 982 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.67:39441
I20260810 07:17:41.901104 1122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.67:39441 every 8 connection(s)
I20260810 07:17:41.901448 982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
Aug 10 07:17:41 dist-test-slave-10pc krb5kdc[569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346261, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:17:41.910929 662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:41.902234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:50505 (local address 127.15.217.126:35977)
0810 07:17:41.902304 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:41.902306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:41.902496 (+ 190us) server_negotiation.cc:407] Connection header received
0810 07:17:41.903003 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:41.903004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:41.903013 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:41.903097 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:41.904073 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.904662 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.905691 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.905888 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.905945 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:41.908100 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:41.908102 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:41.908102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:41.908110 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:41.909747 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:41.910118 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:41.910119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:41.910119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:41.910174 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:41.910395 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:41.910395 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:41.910395 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:41.910500 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:41.910551 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:41.910796 (+ 245us) server_negotiation.cc:299] Negotiation successful
0810 07:17:41.910862 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 07:17:41.911146 1125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:41.902205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.67:50505)
0810 07:17:41.902365 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:41.902408 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:41.902915 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:41.903156 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:41.903160 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:41.903756 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:41.903965 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:41.903970 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.904759 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.904760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:41.905601 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:41.905604 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.905973 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.905974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:41.906091 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:41.906586 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:41.906595 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:41.907995 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:41.909817 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:41.909824 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:41.909826 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:41.910048 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:41.910234 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:41.910235 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:41.910236 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:41.910339 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:41.910555 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:41.910560 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:41.910681 (+ 121us) client_negotiation.cc:769] Sending connection context
0810 07:17:41.910765 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 07:17:41.910892 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":85,"threads_started":1}
I20260810 07:17:41.911705 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 982
I20260810 07:17:41.911772 1123 heartbeater.cc:347] Connected to a master server at 127.15.217.126:35977
I20260810 07:17:41.911813 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance
I20260810 07:17:41.911886 1123 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:41.912009 16229 external_mini_cluster.cc:1428] Setting key c52fbae1d0920bf8031739fe5d240aa3
I20260810 07:17:41.912132 1123 heartbeater.cc:511] Master 127.15.217.126:35977 requested a full tablet report, sending...
I20260810 07:17:41.912575 617 ts_manager.cc:194] Registered new tserver with Master: c55b7534c4324b52a0ef9fb73002d48c (127.15.217.67:39441)
I20260810 07:17:41.912985 617 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.67@KRBTEST.COM'} at 127.15.217.67:50505
I20260810 07:17:41.913005 16229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:17:41.920287 662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:41.914474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34576 (local address 127.15.217.126:35977)
0810 07:17:41.914512 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:41.914514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:41.914652 (+ 138us) server_negotiation.cc:407] Connection header received
0810 07:17:41.914702 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:41.914702 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:41.914712 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:41.914793 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:41.915447 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.915899 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.916689 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.916867 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.916920 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:41.917963 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:41.917964 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:41.917964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:41.917970 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:41.919156 (+ 1186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:41.919532 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:41.919533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:41.919534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:41.919608 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:41.919830 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:41.919831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:41.919831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:41.919985 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:41.920092 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:41.920161 (+ 69us) server_negotiation.cc:299] Negotiation successful
0810 07:17:41.920223 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 07:17:41.921952 617 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34576:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:17:41.923568 617 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:17:41.930495 1135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:41.926146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:42967 (local address 127.0.0.1:34236)
0810 07:17:41.926662 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:41.926670 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:41.926714 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:41.926967 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:41.926968 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:41.926970 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:17:41.927027 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:41.927029 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.927974 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.927975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:41.929225 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:41.929228 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.930229 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.930230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:41.930299 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:41.930304 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:17:41.930353 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 07:17:41.930399 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":105,"threads_started":1}
I20260810 07:17:41.930521 1139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:41.926441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52026 (local address 127.15.217.66:36221)
0810 07:17:41.926673 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:41.926677 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:41.926687 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:17:41.926727 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:41.926728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:41.926741 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:41.926840 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:17:41.927110 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.928244 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.929518 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.930002 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.930110 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:41.930176 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:41.930353 (+ 177us) server_negotiation.cc:299] Negotiation successful
0810 07:17:41.930397 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":112,"threads_started":1}
I20260810 07:17:41.930516 1134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:41.926144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:36221 (local address 127.0.0.1:52026)
0810 07:17:41.926435 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:41.926480 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:41.926549 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:41.926941 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:41.926944 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:41.926947 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:17:41.927022 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:41.927024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.928355 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.928356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:41.929415 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:41.929417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.930115 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.930117 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:41.930209 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:41.930223 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 07:17:41.930325 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 07:17:41.930422 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":2,"thread_start_us":80,"threads_started":1}
I20260810 07:17:41.930651 1136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:41.926152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:39441 (local address 127.0.0.1:42246)
0810 07:17:41.926812 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:41.926823 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:41.926865 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:41.927151 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:41.927151 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:41.927153 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:17:41.927193 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:41.927195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.928413 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.928414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:41.929455 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:41.929458 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.930431 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.930432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:41.930493 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:41.930496 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:17:41.930546 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 07:17:41.930588 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"mutex_wait_us":12,"thread_start_us":83,"threads_started":1}
I20260810 07:17:41.930814 1137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:41.926256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34236 (local address 127.15.217.65:42967)
0810 07:17:41.926628 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:41.926633 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:41.926707 (+ 74us) server_negotiation.cc:407] Connection header received
0810 07:17:41.926779 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:41.926782 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:41.926800 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:41.926880 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:17:41.927110 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.927868 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.929318 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.930045 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.930119 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:41.930187 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:41.930640 (+ 453us) server_negotiation.cc:299] Negotiation successful
0810 07:17:41.930706 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":76,"threads_started":1}
I20260810 07:17:41.930940 1138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:41.926418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42246 (local address 127.15.217.67:39441)
0810 07:17:41.926610 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:41.926616 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:41.926922 (+ 306us) server_negotiation.cc:407] Connection header received
0810 07:17:41.926967 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:41.926969 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:41.926983 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:41.927051 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:17:41.927278 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.928326 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.929553 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.930255 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.930343 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:41.930397 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:41.930811 (+ 414us) server_negotiation.cc:299] Negotiation successful
0810 07:17:41.930861 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":94,"threads_started":1}
I20260810 07:17:41.931726 1051 tablet_service.cc:1467] Processing CreateTablet for tablet 6e7590e75bae4303a9f38ce3cb79c8a1 (DEFAULT_TABLE table=test-table [id=edcd378c61434268afe5de63b7035f80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:17:41.931856 759 tablet_service.cc:1467] Processing CreateTablet for tablet 6e7590e75bae4303a9f38ce3cb79c8a1 (DEFAULT_TABLE table=test-table [id=edcd378c61434268afe5de63b7035f80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:17:41.932124 1051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e7590e75bae4303a9f38ce3cb79c8a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:17:41.932068 906 tablet_service.cc:1467] Processing CreateTablet for tablet 6e7590e75bae4303a9f38ce3cb79c8a1 (DEFAULT_TABLE table=test-table [id=edcd378c61434268afe5de63b7035f80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:17:41.932190 759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e7590e75bae4303a9f38ce3cb79c8a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:17:41.932365 906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e7590e75bae4303a9f38ce3cb79c8a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:17:41.935809 1140 tablet_bootstrap.cc:492] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 7a71dca4901f4ab4b2bc099682fb3ec2: Bootstrap starting.
I20260810 07:17:41.936057 1141 tablet_bootstrap.cc:492] T 6e7590e75bae4303a9f38ce3cb79c8a1 P c55b7534c4324b52a0ef9fb73002d48c: Bootstrap starting.
I20260810 07:17:41.936779 1141 tablet_bootstrap.cc:654] T 6e7590e75bae4303a9f38ce3cb79c8a1 P c55b7534c4324b52a0ef9fb73002d48c: Neither blocks nor log segments found. Creating new log.
I20260810 07:17:41.937116 1140 tablet_bootstrap.cc:654] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 7a71dca4901f4ab4b2bc099682fb3ec2: Neither blocks nor log segments found. Creating new log.
I20260810 07:17:41.937342 1141 log.cc:824] T 6e7590e75bae4303a9f38ce3cb79c8a1 P c55b7534c4324b52a0ef9fb73002d48c: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:41.937753 1140 log.cc:824] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 7a71dca4901f4ab4b2bc099682fb3ec2: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:41.938099 1142 tablet_bootstrap.cc:492] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 308b655a3d4d4ba793d2ac7f4e74cd92: Bootstrap starting.
I20260810 07:17:41.938301 1141 tablet_bootstrap.cc:492] T 6e7590e75bae4303a9f38ce3cb79c8a1 P c55b7534c4324b52a0ef9fb73002d48c: No bootstrap required, opened a new log
I20260810 07:17:41.938381 1141 ts_tablet_manager.cc:1397] T 6e7590e75bae4303a9f38ce3cb79c8a1 P c55b7534c4324b52a0ef9fb73002d48c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260810 07:17:41.938501 1140 tablet_bootstrap.cc:492] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 7a71dca4901f4ab4b2bc099682fb3ec2: No bootstrap required, opened a new log
I20260810 07:17:41.938612 1140 ts_tablet_manager.cc:1397] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 7a71dca4901f4ab4b2bc099682fb3ec2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:17:41.938798 1142 tablet_bootstrap.cc:654] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 308b655a3d4d4ba793d2ac7f4e74cd92: Neither blocks nor log segments found. Creating new log.
I20260810 07:17:41.939296 1142 log.cc:824] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 308b655a3d4d4ba793d2ac7f4e74cd92: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:41.940006 1142 tablet_bootstrap.cc:492] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 308b655a3d4d4ba793d2ac7f4e74cd92: No bootstrap required, opened a new log
I20260810 07:17:41.940093 1142 ts_tablet_manager.cc:1397] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 308b655a3d4d4ba793d2ac7f4e74cd92: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:17:41.940492 1141 raft_consensus.cc:348] T 6e7590e75bae4303a9f38ce3cb79c8a1 P c55b7534c4324b52a0ef9fb73002d48c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 36221 } } peers { permanent_uuid: "7a71dca4901f4ab4b2bc099682fb3ec2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 42967 } } peers { permanent_uuid: "c55b7534c4324b52a0ef9fb73002d48c" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39441 } }
I20260810 07:17:41.940691 1141 raft_consensus.cc:374] T 6e7590e75bae4303a9f38ce3cb79c8a1 P c55b7534c4324b52a0ef9fb73002d48c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:17:41.940763 1141 raft_consensus.cc:729] T 6e7590e75bae4303a9f38ce3cb79c8a1 P c55b7534c4324b52a0ef9fb73002d48c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c55b7534c4324b52a0ef9fb73002d48c, State: Initialized, Role: FOLLOWER
I20260810 07:17:41.940675 1140 raft_consensus.cc:348] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 36221 } } peers { permanent_uuid: "7a71dca4901f4ab4b2bc099682fb3ec2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 42967 } } peers { permanent_uuid: "c55b7534c4324b52a0ef9fb73002d48c" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39441 } }
I20260810 07:17:41.940879 1140 raft_consensus.cc:374] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:17:41.940900 1141 consensus_queue.cc:260] T 6e7590e75bae4303a9f38ce3cb79c8a1 P c55b7534c4324b52a0ef9fb73002d48c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 36221 } } peers { permanent_uuid: "7a71dca4901f4ab4b2bc099682fb3ec2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 42967 } } peers { permanent_uuid: "c55b7534c4324b52a0ef9fb73002d48c" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39441 } }
I20260810 07:17:41.940954 1140 raft_consensus.cc:729] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a71dca4901f4ab4b2bc099682fb3ec2, State: Initialized, Role: FOLLOWER
I20260810 07:17:41.941092 1140 consensus_queue.cc:260] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 7a71dca4901f4ab4b2bc099682fb3ec2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 36221 } } peers { permanent_uuid: "7a71dca4901f4ab4b2bc099682fb3ec2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 42967 } } peers { permanent_uuid: "c55b7534c4324b52a0ef9fb73002d48c" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39441 } }
I20260810 07:17:41.941216 1141 ts_tablet_manager.cc:1428] T 6e7590e75bae4303a9f38ce3cb79c8a1 P c55b7534c4324b52a0ef9fb73002d48c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:17:41.941253 1123 heartbeater.cc:503] Master 127.15.217.126:35977 was elected leader, sending a full tablet report...
I20260810 07:17:41.941478 1140 ts_tablet_manager.cc:1428] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 7a71dca4901f4ab4b2bc099682fb3ec2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:17:41.941639 832 heartbeater.cc:503] Master 127.15.217.126:35977 was elected leader, sending a full tablet report...
I20260810 07:17:41.942819 1142 raft_consensus.cc:348] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 308b655a3d4d4ba793d2ac7f4e74cd92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 36221 } } peers { permanent_uuid: "7a71dca4901f4ab4b2bc099682fb3ec2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 42967 } } peers { permanent_uuid: "c55b7534c4324b52a0ef9fb73002d48c" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39441 } }
I20260810 07:17:41.942955 1142 raft_consensus.cc:374] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 308b655a3d4d4ba793d2ac7f4e74cd92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:17:41.942978 1142 raft_consensus.cc:729] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 308b655a3d4d4ba793d2ac7f4e74cd92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 308b655a3d4d4ba793d2ac7f4e74cd92, State: Initialized, Role: FOLLOWER
I20260810 07:17:41.943112 1142 consensus_queue.cc:260] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 308b655a3d4d4ba793d2ac7f4e74cd92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 36221 } } peers { permanent_uuid: "7a71dca4901f4ab4b2bc099682fb3ec2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 42967 } } peers { permanent_uuid: "c55b7534c4324b52a0ef9fb73002d48c" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39441 } }
I20260810 07:17:41.943418 1142 ts_tablet_manager.cc:1428] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 308b655a3d4d4ba793d2ac7f4e74cd92: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:17:41.943457 977 heartbeater.cc:503] Master 127.15.217.126:35977 was elected leader, sending a full tablet report...
W20260810 07:17:41.993571 978 tablet.cc:2369] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 308b655a3d4d4ba793d2ac7f4e74cd92: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:17:42.036672 1148 raft_consensus.cc:482] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 308b655a3d4d4ba793d2ac7f4e74cd92 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:17:42.036862 1148 raft_consensus.cc:504] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 308b655a3d4d4ba793d2ac7f4e74cd92 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 36221 } } peers { permanent_uuid: "7a71dca4901f4ab4b2bc099682fb3ec2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 42967 } } peers { permanent_uuid: "c55b7534c4324b52a0ef9fb73002d48c" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39441 } }
I20260810 07:17:42.037295 1148 leader_election.cc:288] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 308b655a3d4d4ba793d2ac7f4e74cd92 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a71dca4901f4ab4b2bc099682fb3ec2 (127.15.217.65:42967), c55b7534c4324b52a0ef9fb73002d48c (127.15.217.67:39441)
I20260810 07:17:42.041298 1138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:42.037650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:48399 (local address 127.15.217.67:39441)
0810 07:17:42.037712 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:42.037714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:42.037727 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:17:42.037762 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:42.037763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:42.037780 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:42.037830 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:17:42.038047 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:42.038830 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:42.040294 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:42.040910 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:42.040975 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:42.040984 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:42.041177 (+ 193us) server_negotiation.cc:299] Negotiation successful
0810 07:17:42.041222 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 07:17:42.041298 1137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:42.037591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:46497 (local address 127.15.217.65:42967)
0810 07:17:42.037639 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:42.037641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:42.037911 (+ 270us) server_negotiation.cc:407] Connection header received
0810 07:17:42.037942 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:42.037944 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:42.037958 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:42.038028 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:17:42.038211 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:42.038892 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:42.040294 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:42.040881 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:42.040967 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:42.040978 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:42.041189 (+ 211us) server_negotiation.cc:299] Negotiation successful
0810 07:17:42.041230 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 07:17:42.041302 979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:42.037496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:39441 (local address 127.15.217.66:48399)
0810 07:17:42.037573 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:42.037588 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:42.037659 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:42.037865 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:42.037866 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:42.037868 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:17:42.037965 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:42.037967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:42.038918 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:42.038919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:42.040193 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:42.040196 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:42.040980 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:42.040982 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:42.041075 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:42.041079 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:17:42.041145 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 07:17:42.041197 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":6}
I20260810 07:17:42.041306 1149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:42.037496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:42967 (local address 127.15.217.66:46497)
0810 07:17:42.037797 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:42.037806 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:42.037881 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:42.038046 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:42.038047 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:42.038049 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:17:42.038130 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:42.038132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:42.038978 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:42.038979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:42.040193 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:42.040196 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:42.040981 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:42.040982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:42.041095 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:42.041098 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:17:42.041153 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 07:17:42.041197 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":119,"thread_start_us":142,"threads_started":1}
I20260810 07:17:42.041716 1073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e7590e75bae4303a9f38ce3cb79c8a1" candidate_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c55b7534c4324b52a0ef9fb73002d48c" is_pre_election: true
I20260810 07:17:42.041940 1073 raft_consensus.cc:2454] T 6e7590e75bae4303a9f38ce3cb79c8a1 P c55b7534c4324b52a0ef9fb73002d48c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 308b655a3d4d4ba793d2ac7f4e74cd92 in term 0.
I20260810 07:17:42.042019 783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e7590e75bae4303a9f38ce3cb79c8a1" candidate_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a71dca4901f4ab4b2bc099682fb3ec2" is_pre_election: true
I20260810 07:17:42.042196 783 raft_consensus.cc:2454] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 308b655a3d4d4ba793d2ac7f4e74cd92 in term 0.
I20260810 07:17:42.042204 856 leader_election.cc:302] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 308b655a3d4d4ba793d2ac7f4e74cd92 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 308b655a3d4d4ba793d2ac7f4e74cd92, c55b7534c4324b52a0ef9fb73002d48c; no voters:
I20260810 07:17:42.042361 1148 raft_consensus.cc:2781] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 308b655a3d4d4ba793d2ac7f4e74cd92 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:17:42.042419 1148 raft_consensus.cc:482] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 308b655a3d4d4ba793d2ac7f4e74cd92 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:17:42.042456 1148 raft_consensus.cc:3037] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 308b655a3d4d4ba793d2ac7f4e74cd92 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:17:42.043202 1148 raft_consensus.cc:504] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 308b655a3d4d4ba793d2ac7f4e74cd92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 36221 } } peers { permanent_uuid: "7a71dca4901f4ab4b2bc099682fb3ec2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 42967 } } peers { permanent_uuid: "c55b7534c4324b52a0ef9fb73002d48c" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39441 } }
I20260810 07:17:42.043359 1148 leader_election.cc:288] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 308b655a3d4d4ba793d2ac7f4e74cd92 [CANDIDATE]: Term 1 election: Requested vote from peers 7a71dca4901f4ab4b2bc099682fb3ec2 (127.15.217.65:42967), c55b7534c4324b52a0ef9fb73002d48c (127.15.217.67:39441)
I20260810 07:17:42.043510 783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e7590e75bae4303a9f38ce3cb79c8a1" candidate_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a71dca4901f4ab4b2bc099682fb3ec2"
I20260810 07:17:42.043533 1073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e7590e75bae4303a9f38ce3cb79c8a1" candidate_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c55b7534c4324b52a0ef9fb73002d48c"
I20260810 07:17:42.043609 1073 raft_consensus.cc:3037] T 6e7590e75bae4303a9f38ce3cb79c8a1 P c55b7534c4324b52a0ef9fb73002d48c [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:17:42.043613 783 raft_consensus.cc:3037] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:17:42.044337 1073 raft_consensus.cc:2454] T 6e7590e75bae4303a9f38ce3cb79c8a1 P c55b7534c4324b52a0ef9fb73002d48c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 308b655a3d4d4ba793d2ac7f4e74cd92 in term 1.
I20260810 07:17:42.044507 783 raft_consensus.cc:2454] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 308b655a3d4d4ba793d2ac7f4e74cd92 in term 1.
I20260810 07:17:42.044531 856 leader_election.cc:302] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 308b655a3d4d4ba793d2ac7f4e74cd92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 308b655a3d4d4ba793d2ac7f4e74cd92, c55b7534c4324b52a0ef9fb73002d48c; no voters:
I20260810 07:17:42.044608 1148 raft_consensus.cc:2781] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 308b655a3d4d4ba793d2ac7f4e74cd92 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:17:42.044785 1148 raft_consensus.cc:686] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 308b655a3d4d4ba793d2ac7f4e74cd92 [term 1 LEADER]: Becoming Leader. State: Replica: 308b655a3d4d4ba793d2ac7f4e74cd92, State: Running, Role: LEADER
I20260810 07:17:42.044888 1148 consensus_queue.cc:237] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 308b655a3d4d4ba793d2ac7f4e74cd92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 36221 } } peers { permanent_uuid: "7a71dca4901f4ab4b2bc099682fb3ec2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 42967 } } peers { permanent_uuid: "c55b7534c4324b52a0ef9fb73002d48c" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39441 } }
I20260810 07:17:42.045794 616 catalog_manager.cc:5495] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 308b655a3d4d4ba793d2ac7f4e74cd92 reported cstate change: term changed from 0 to 1, leader changed from <none> to 308b655a3d4d4ba793d2ac7f4e74cd92 (127.15.217.66). New cstate: current_term: 1 leader_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 36221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a71dca4901f4ab4b2bc099682fb3ec2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 42967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c55b7534c4324b52a0ef9fb73002d48c" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39441 } health_report { overall_health: UNKNOWN } } }
W20260810 07:17:42.070235 834 tablet.cc:2369] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 7a71dca4901f4ab4b2bc099682fb3ec2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:17:42.075654 1139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:42.073126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52032 (local address 127.15.217.66:36221)
0810 07:17:42.073194 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:42.073196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:42.073204 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:17:42.073244 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:42.073245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:42.073268 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:42.073351 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:17:42.073506 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:42.073993 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:42.074793 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:42.074997 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:42.075077 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:42.075274 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:17:42.075387 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:17:42.075489 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:42.075508 (+ 19us) server_negotiation.cc:299] Negotiation successful
0810 07:17:42.075531 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 07:17:42.077805 1073 raft_consensus.cc:1261] T 6e7590e75bae4303a9f38ce3cb79c8a1 P c55b7534c4324b52a0ef9fb73002d48c [term 1 FOLLOWER]: Refusing update from remote peer 308b655a3d4d4ba793d2ac7f4e74cd92: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:17:42.077823 783 raft_consensus.cc:1261] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 1 FOLLOWER]: Refusing update from remote peer 308b655a3d4d4ba793d2ac7f4e74cd92: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:17:42.078214 1150 consensus_queue.cc:1035] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 308b655a3d4d4ba793d2ac7f4e74cd92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c55b7534c4324b52a0ef9fb73002d48c" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:17:42.078348 1148 consensus_queue.cc:1035] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 308b655a3d4d4ba793d2ac7f4e74cd92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a71dca4901f4ab4b2bc099682fb3ec2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:17:42.081013 1156 mvcc.cc:205] Tried to move back new op lower bound from 7316874289467580416 to 7316874289336532992. Current Snapshot: MvccSnapshot[applied={T|T < 7316874289467580416}]
I20260810 07:17:42.082094 1154 mvcc.cc:205] Tried to move back new op lower bound from 7316874289467580416 to 7316874289336532992. Current Snapshot: MvccSnapshot[applied={T|T < 7316874289467580416}]
I20260810 07:17:42.083181 616 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:17:42.083276 616 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:17:42.084110 616 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1: Sending DeleteTablet for 3 replicas of tablet 6e7590e75bae4303a9f38ce3cb79c8a1
I20260810 07:17:42.084482 906 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e7590e75bae4303a9f38ce3cb79c8a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:17:42 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:52026
I20260810 07:17:42.084481 1051 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e7590e75bae4303a9f38ce3cb79c8a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:17:42 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:42246
I20260810 07:17:42.084525 759 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e7590e75bae4303a9f38ce3cb79c8a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:17:42 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:34236
I20260810 07:17:42.084760 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 689
I20260810 07:17:42.084846 1166 tablet_replica.cc:330] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 308b655a3d4d4ba793d2ac7f4e74cd92: stopping tablet replica
I20260810 07:17:42.087054 1166 raft_consensus.cc:2229] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 308b655a3d4d4ba793d2ac7f4e74cd92 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:17:42.087216 1166 raft_consensus.cc:2258] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 308b655a3d4d4ba793d2ac7f4e74cd92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:17:42.087890 1166 ts_tablet_manager.cc:1905] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 308b655a3d4d4ba793d2ac7f4e74cd92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:17:42.089752 1165 tablet_replica.cc:330] T 6e7590e75bae4303a9f38ce3cb79c8a1 P c55b7534c4324b52a0ef9fb73002d48c: stopping tablet replica
I20260810 07:17:42.089936 1165 raft_consensus.cc:2229] T 6e7590e75bae4303a9f38ce3cb79c8a1 P c55b7534c4324b52a0ef9fb73002d48c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:17:42.090112 1165 raft_consensus.cc:2258] T 6e7590e75bae4303a9f38ce3cb79c8a1 P c55b7534c4324b52a0ef9fb73002d48c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:17:42.090727 1165 ts_tablet_manager.cc:1905] T 6e7590e75bae4303a9f38ce3cb79c8a1 P c55b7534c4324b52a0ef9fb73002d48c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:17:42.090956 1166 ts_tablet_manager.cc:1918] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 308b655a3d4d4ba793d2ac7f4e74cd92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:17:42.091091 1166 log.cc:1196] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 308b655a3d4d4ba793d2ac7f4e74cd92: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/wal/wals/6e7590e75bae4303a9f38ce3cb79c8a1
I20260810 07:17:42.091465 1166 ts_tablet_manager.cc:1939] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 308b655a3d4d4ba793d2ac7f4e74cd92: Deleting consensus metadata
I20260810 07:17:42.092049 602 catalog_manager.cc:4841] TS 308b655a3d4d4ba793d2ac7f4e74cd92 (127.15.217.66:36221): tablet 6e7590e75bae4303a9f38ce3cb79c8a1 (table test-table [id=edcd378c61434268afe5de63b7035f80]) successfully deleted
I20260810 07:17:42.092607 1165 ts_tablet_manager.cc:1918] T 6e7590e75bae4303a9f38ce3cb79c8a1 P c55b7534c4324b52a0ef9fb73002d48c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:17:42.092715 1165 log.cc:1196] T 6e7590e75bae4303a9f38ce3cb79c8a1 P c55b7534c4324b52a0ef9fb73002d48c: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/wal/wals/6e7590e75bae4303a9f38ce3cb79c8a1
I20260810 07:17:42.092998 1165 ts_tablet_manager.cc:1939] T 6e7590e75bae4303a9f38ce3cb79c8a1 P c55b7534c4324b52a0ef9fb73002d48c: Deleting consensus metadata
I20260810 07:17:42.093464 601 catalog_manager.cc:4841] TS c55b7534c4324b52a0ef9fb73002d48c (127.15.217.67:39441): tablet 6e7590e75bae4303a9f38ce3cb79c8a1 (table test-table [id=edcd378c61434268afe5de63b7035f80]) successfully deleted
W20260810 07:17:42.096114 603 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.217.65:42967 (error 108)
I20260810 07:17:42.096724 1134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:42.096563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:42967 (local address 127.0.0.1:34246)
0810 07:17:42.096619 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:42.096658 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.65:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260810 07:17:42.096794 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 838
W20260810 07:17:42.096865 603 catalog_manager.cc:4584] TS 7a71dca4901f4ab4b2bc099682fb3ec2 (127.15.217.65:42967): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6e7590e75bae4303a9f38ce3cb79c8a1: Network error: Client connection negotiation failed: client connection to 127.15.217.65:42967: connect: Connection refused (error 111)
I20260810 07:17:42.113230 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 982
I20260810 07:17:42.120184 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 582
I20260810 07:18:02.127847 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:35977
--webserver_interface=127.15.217.126
--webserver_port=38241
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:33041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:35977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:18:02.215359 1169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:02.215626 1169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:02.215668 1169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:02.216990 1169 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 07:18:02.217065 1169 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 07:18:02.217088 1169 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 07:18:02.217108 1169 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:18:02.217149 1169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:02.217177 1169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:02.217203 1169 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:18:02.217231 1169 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:18:02.218868 1169 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:33041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:35977
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:35977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=38241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:02.219202 1169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:02.219460 1169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:02.230520 1176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:02.230542 1169 server_base.cc:1034] running on GCE node
W20260810 07:18:02.230543 1177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:02.230531 1179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:02.231235 1169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:02.231601 1169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:02.232772 1169 hybrid_clock.cc:648] HybridClock initialized: now 1786346282232759 us; error 28 us; skew 500 ppm
Aug 10 07:18:02 dist-test-slave-10pc krb5kdc[569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346282, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:02.242118 1169 init.cc:377] Logged in from keytab as kudu/127.15.217.126@KRBTEST.COM (short username kudu)
I20260810 07:18:02.242781 1169 webserver.cc:460] Webserver started at http://127.15.217.126:38241/ using document root <none> and password file <none>
I20260810 07:18:02.243139 1169 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:02.243227 1169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:02.245213 1169 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:18:02.246124 1187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:02.246376 1169 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:18:02.246490 1169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "1928338c4bf347c1b69f77261844dca1"
format_stamp: "Formatted at 2026-08-10 07:17:41 on dist-test-slave-10pc"
server_key: "59269af0ebc5c8a76e00fa02e1bc9eb8"
server_key_iv: "88f9a2ec389d8b71ba46eb8e610de5d2"
server_key_version: "encryptionkey@0"
I20260810 07:18:02.247145 1169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:02.262835 1169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:02.265067 1169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:02.265231 1169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:02.269595 1169 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:35977
I20260810 07:18:02.269663 1243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:35977 every 8 connection(s)
I20260810 07:18:02.270097 1169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:18:02.271745 1244 sys_catalog.cc:263] Verifying existing consensus state
I20260810 07:18:02.272547 1244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1: Bootstrap starting.
I20260810 07:18:02.274820 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 1169
I20260810 07:18:02.275185 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.65:42967
--local_ip_for_outbound_sockets=127.15.217.65
--tserver_master_addrs=127.15.217.126:35977
--webserver_port=36741
--webserver_interface=127.15.217.65
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:33041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:18:02.282939 1244 log.cc:824] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:02.285302 1244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:18:02.285670 1244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1: Bootstrap complete.
I20260810 07:18:02.288209 1244 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1928338c4bf347c1b69f77261844dca1" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35977 } }
I20260810 07:18:02.288527 1244 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1928338c4bf347c1b69f77261844dca1, State: Initialized, Role: FOLLOWER
I20260810 07:18:02.288748 1244 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1928338c4bf347c1b69f77261844dca1" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35977 } }
I20260810 07:18:02.288849 1244 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:18:02.288911 1244 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:18:02.288962 1244 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:18:02.290119 1244 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1928338c4bf347c1b69f77261844dca1" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35977 } }
I20260810 07:18:02.290238 1244 leader_election.cc:302] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1928338c4bf347c1b69f77261844dca1; no voters:
I20260810 07:18:02.290449 1244 leader_election.cc:288] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 07:18:02.290491 1250 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 07:18:02.290632 1250 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [term 2 LEADER]: Becoming Leader. State: Replica: 1928338c4bf347c1b69f77261844dca1, State: Running, Role: LEADER
I20260810 07:18:02.290764 1244 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:18:02.290799 1250 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1928338c4bf347c1b69f77261844dca1" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35977 } }
I20260810 07:18:02.291118 1250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1928338c4bf347c1b69f77261844dca1. Latest consensus state: current_term: 2 leader_uuid: "1928338c4bf347c1b69f77261844dca1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1928338c4bf347c1b69f77261844dca1" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35977 } } }
I20260810 07:18:02.291126 1252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1928338c4bf347c1b69f77261844dca1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1928338c4bf347c1b69f77261844dca1" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35977 } } }
I20260810 07:18:02.291405 1252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [sys.catalog]: This master's current role is: LEADER
I20260810 07:18:02.291328 1250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [sys.catalog]: This master's current role is: LEADER
I20260810 07:18:02.291667 1260 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:18:02.292701 1260 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:18:02.292867 1260 catalog_manager.cc:1259] Loaded cluster ID: 721586d3450246ceabce0cda93082d71
I20260810 07:18:02.293151 1260 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:18:02.294361 1260 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:18:02.294561 1260 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1: Loaded TSK: 0
I20260810 07:18:02.305958 1260 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1: Generated new TSK 1
I20260810 07:18:02.306100 1260 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 07:18:02.365556 1247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:02.365808 1247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:02.365849 1247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:02.367024 1247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:02.367097 1247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:02.367142 1247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.65
I20260810 07:18:02.368686 1247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:33041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.65:42967
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.217.65
--webserver_port=36741
--tserver_master_addrs=127.15.217.126:35977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.65
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:02.368988 1247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:02.369238 1247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:02.375567 1277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:02.375607 1276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:02.375607 1279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:02.376264 1247 server_base.cc:1034] running on GCE node
I20260810 07:18:02.376529 1247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:02.376821 1247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:02.377986 1247 hybrid_clock.cc:648] HybridClock initialized: now 1786346282377964 us; error 38 us; skew 500 ppm
Aug 10 07:18:02 dist-test-slave-10pc krb5kdc[569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346282, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:02.380739 1247 init.cc:377] Logged in from keytab as kudu/127.15.217.65@KRBTEST.COM (short username kudu)
I20260810 07:18:02.381283 1247 webserver.cc:460] Webserver started at http://127.15.217.65:36741/ using document root <none> and password file <none>
I20260810 07:18:02.381579 1247 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:02.381687 1247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:02.383208 1247 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:18:02.383917 1287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:02.384083 1247 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 07:18:02.384196 1247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/wal
uuid: "7a71dca4901f4ab4b2bc099682fb3ec2"
format_stamp: "Formatted at 2026-08-10 07:17:41 on dist-test-slave-10pc"
server_key: "66148bb75b5517b2d9382b42069e2b3e"
server_key_iv: "eb0ea3b6d15a7b8715ac91d65b6a307a"
server_key_version: "encryptionkey@0"
I20260810 07:18:02.384647 1247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:02.423866 1247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:02.426126 1247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:02.426297 1247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:02.426558 1247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:18:02.427047 1294 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 07:18:02.428635 1247 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 07:18:02.428696 1247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260810 07:18:02.428754 1247 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 07:18:02.429464 1247 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 07:18:02.429520 1247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260810 07:18:02.429600 1294 tablet_bootstrap.cc:492] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 7a71dca4901f4ab4b2bc099682fb3ec2: Bootstrap starting.
I20260810 07:18:02.437289 1247 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.65:42967
I20260810 07:18:02.437366 1415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.65:42967 every 8 connection(s)
I20260810 07:18:02.437750 1247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
I20260810 07:18:02.439764 1294 log.cc:824] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 7a71dca4901f4ab4b2bc099682fb3ec2: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:02.442165 1294 tablet_bootstrap.cc:492] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 7a71dca4901f4ab4b2bc099682fb3ec2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:18:02.442502 1294 tablet_bootstrap.cc:492] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 7a71dca4901f4ab4b2bc099682fb3ec2: Bootstrap complete.
I20260810 07:18:02.442613 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 1247
I20260810 07:18:02.442911 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.66:36221
--local_ip_for_outbound_sockets=127.15.217.66
--tserver_master_addrs=127.15.217.126:35977
--webserver_port=45335
--webserver_interface=127.15.217.66
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:33041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:18:02.443746 1294 ts_tablet_manager.cc:1397] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 7a71dca4901f4ab4b2bc099682fb3ec2: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.008s
I20260810 07:18:02.445220 1294 raft_consensus.cc:348] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 36221 } } peers { permanent_uuid: "7a71dca4901f4ab4b2bc099682fb3ec2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 42967 } } peers { permanent_uuid: "c55b7534c4324b52a0ef9fb73002d48c" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39441 } }
I20260810 07:18:02.445570 1294 raft_consensus.cc:729] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a71dca4901f4ab4b2bc099682fb3ec2, State: Initialized, Role: FOLLOWER
I20260810 07:18:02.445767 1294 consensus_queue.cc:260] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 7a71dca4901f4ab4b2bc099682fb3ec2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 36221 } } peers { permanent_uuid: "7a71dca4901f4ab4b2bc099682fb3ec2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 42967 } } peers { permanent_uuid: "c55b7534c4324b52a0ef9fb73002d48c" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39441 } }
I20260810 07:18:02.446084 1294 ts_tablet_manager.cc:1428] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 7a71dca4901f4ab4b2bc099682fb3ec2: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
Aug 10 07:18:02 dist-test-slave-10pc krb5kdc[569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346282, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:18:02.450783 1419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:02.438644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:34147 (local address 127.15.217.126:35977)
0810 07:18:02.438819 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:02.438828 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:02.438890 (+ 62us) server_negotiation.cc:407] Connection header received
0810 07:18:02.439751 (+ 861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:02.439757 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:02.439802 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:02.439943 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:02.440778 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.441683 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.443313 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.443949 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.444064 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:02.446800 (+ 2736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:02.446804 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:02.446804 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:02.446817 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:02.448752 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:02.449278 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:02.449287 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:02.449287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:02.449405 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:02.449707 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:02.449708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:02.449708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:02.449960 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:02.450060 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:02.450417 (+ 357us) server_negotiation.cc:299] Negotiation successful
0810 07:18:02.450569 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":94,"threads_started":1}
I20260810 07:18:02.450783 1418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:02.438479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.65:34147)
0810 07:18:02.438702 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:02.438783 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:02.439527 (+ 744us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:02.440029 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:02.440034 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:02.440328 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:18:02.440645 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:02.440652 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.441813 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.441814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:02.443219 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:02.443223 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.444059 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.444061 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:02.444205 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:02.444860 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:18:02.444881 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:18:02.446684 (+ 1803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:18:02.448881 (+ 2197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:02.448890 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:02.448892 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:02.449168 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:02.449506 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:02.449507 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:02.449507 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:02.449623 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:02.450050 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:18:02.450058 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:18:02.450235 (+ 177us) client_negotiation.cc:769] Sending connection context
0810 07:18:02.450354 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 07:18:02.450565 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":108,"threads_started":1}
I20260810 07:18:02.451907 1416 heartbeater.cc:347] Connected to a master server at 127.15.217.126:35977
I20260810 07:18:02.452033 1416 heartbeater.cc:464] Registering TS with master...
I20260810 07:18:02.452241 1416 heartbeater.cc:511] Master 127.15.217.126:35977 requested a full tablet report, sending...
I20260810 07:18:02.453068 1205 ts_manager.cc:194] Registered new tserver with Master: 7a71dca4901f4ab4b2bc099682fb3ec2 (127.15.217.65:42967)
I20260810 07:18:02.454308 1205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.65@KRBTEST.COM'} at 127.15.217.65:34147
I20260810 07:18:02.455426 1416 heartbeater.cc:503] Master 127.15.217.126:35977 was elected leader, sending a full tablet report...
Aug 10 07:18:02 dist-test-slave-10pc krb5kdc[569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346282, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for kudu/127.15.217.65@KRBTEST.COM
I20260810 07:18:02.462586 1424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:02.453715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:42967 (local address 127.0.0.1:38402)
0810 07:18:02.453903 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:02.453954 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:02.454038 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:02.454372 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:02.454377 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:02.454741 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:18:02.454804 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:02.454807 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.455841 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.455842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:02.456820 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:02.456823 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.457306 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.457307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:02.457387 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:02.457845 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:18:02.457850 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:18:02.459402 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:18:02.461383 (+ 1981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:02.461392 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:02.461394 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:02.461608 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:02.461840 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:02.461841 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:02.461841 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:02.461904 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:02.462193 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:18:02.462200 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:18:02.462255 (+ 55us) client_negotiation.cc:769] Sending connection context
0810 07:18:02.462349 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 07:18:02.462485 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":87,"threads_started":1}
I20260810 07:18:02.462587 1425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:02.453984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38402 (local address 127.15.217.65:42967)
0810 07:18:02.454199 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:02.454205 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:02.454216 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:18:02.454254 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:02.454256 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:02.454272 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:02.454349 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:02.454909 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.455736 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.456894 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.457228 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.457294 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:02.459510 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:02.459519 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:02.459524 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:02.459532 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:02.461298 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:02.461705 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:02.461708 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:02.461708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:02.461768 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:02.461967 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:02.461968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:02.461968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:02.462116 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:02.462201 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:02.462393 (+ 192us) server_negotiation.cc:299] Negotiation successful
0810 07:18:02.462490 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":85,"threads_started":1}
I20260810 07:18:02.463016 1340 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e7590e75bae4303a9f38ce3cb79c8a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:17:42 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:38402
I20260810 07:18:02.463557 1426 tablet_replica.cc:330] stopping tablet replica
I20260810 07:18:02.463673 1426 raft_consensus.cc:2229] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:18:02.463745 1426 raft_consensus.cc:2258] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:18:02.464080 1426 ts_tablet_manager.cc:1905] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 7a71dca4901f4ab4b2bc099682fb3ec2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:18:02.466889 1426 ts_tablet_manager.cc:1918] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 7a71dca4901f4ab4b2bc099682fb3ec2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:18:02.466981 1426 log.cc:1196] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 7a71dca4901f4ab4b2bc099682fb3ec2: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/wal/wals/6e7590e75bae4303a9f38ce3cb79c8a1
I20260810 07:18:02.467240 1426 ts_tablet_manager.cc:1939] T 6e7590e75bae4303a9f38ce3cb79c8a1 P 7a71dca4901f4ab4b2bc099682fb3ec2: Deleting consensus metadata
I20260810 07:18:02.467803 1192 catalog_manager.cc:4841] TS 7a71dca4901f4ab4b2bc099682fb3ec2 (127.15.217.65:42967): tablet 6e7590e75bae4303a9f38ce3cb79c8a1 (table test-table [id=edcd378c61434268afe5de63b7035f80]) successfully deleted
W20260810 07:18:02.533875 1421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:02.534096 1421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:02.534119 1421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:02.535283 1421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:02.535327 1421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:02.535356 1421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.66
I20260810 07:18:02.536899 1421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:33041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.66:36221
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.217.66
--webserver_port=45335
--tserver_master_addrs=127.15.217.126:35977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.66
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:02.537143 1421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:02.537374 1421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:02.542505 1432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:02.542497 1434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:02.542465 1431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:02.542543 1421 server_base.cc:1034] running on GCE node
I20260810 07:18:02.542891 1421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:02.543100 1421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:02.544250 1421 hybrid_clock.cc:648] HybridClock initialized: now 1786346282544238 us; error 28 us; skew 500 ppm
Aug 10 07:18:02 dist-test-slave-10pc krb5kdc[569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346282, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:02.547336 1421 init.cc:377] Logged in from keytab as kudu/127.15.217.66@KRBTEST.COM (short username kudu)
I20260810 07:18:02.547820 1421 webserver.cc:460] Webserver started at http://127.15.217.66:45335/ using document root <none> and password file <none>
I20260810 07:18:02.548099 1421 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:02.548192 1421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:02.549768 1421 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:18:02.550475 1441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:02.550666 1421 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:18:02.550752 1421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/wal
uuid: "308b655a3d4d4ba793d2ac7f4e74cd92"
format_stamp: "Formatted at 2026-08-10 07:17:41 on dist-test-slave-10pc"
server_key: "0558f8690cf26aacf559a4a0f1fa565e"
server_key_iv: "cf8e179fb61258221c96cf6df0335485"
server_key_version: "encryptionkey@0"
I20260810 07:18:02.551121 1421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:02.561849 1421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:02.563740 1421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:02.563894 1421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:02.564139 1421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:18:02.564514 1421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:18:02.564566 1421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:02.564589 1421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:18:02.564601 1421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:02.570480 1421 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.66:36221
I20260810 07:18:02.570567 1564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.66:36221 every 8 connection(s)
I20260810 07:18:02.570992 1421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
Aug 10 07:18:02 dist-test-slave-10pc krb5kdc[569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346282, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:18:02.579916 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 1421
I20260810 07:18:02.580322 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.67:39441
--local_ip_for_outbound_sockets=127.15.217.67
--tserver_master_addrs=127.15.217.126:35977
--webserver_port=35797
--webserver_interface=127.15.217.67
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:33041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:18:02.581238 1419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:02.571793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:38107 (local address 127.15.217.126:35977)
0810 07:18:02.571859 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:02.571861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:02.572060 (+ 199us) server_negotiation.cc:407] Connection header received
0810 07:18:02.572651 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:02.572653 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:02.572669 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:02.572726 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:02.573756 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.574291 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.575298 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.575500 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.575551 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:02.578003 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:02.578005 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:02.578005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:02.578012 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:02.579511 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:02.579927 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:02.579929 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:02.579929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:02.579998 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:02.580354 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:02.580356 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:02.580356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:02.580617 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:02.580684 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:02.581073 (+ 389us) server_negotiation.cc:299] Negotiation successful
0810 07:18:02.581166 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 07:18:02.581514 1567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:02.571767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.66:38107)
0810 07:18:02.571934 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:02.571974 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:02.572537 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:02.572826 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:02.572830 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:02.573441 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:18:02.573656 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:02.573661 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.574379 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.574380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:02.575225 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:02.575227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.575606 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.575607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:02.575730 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:02.576273 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:18:02.576284 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:18:02.577890 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:18:02.579589 (+ 1699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:02.579597 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:02.579599 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:02.579849 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:02.580100 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:02.580101 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:02.580102 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:02.580264 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:02.580694 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:18:02.580702 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:18:02.580886 (+ 184us) client_negotiation.cc:769] Sending connection context
0810 07:18:02.581018 (+ 132us) client_negotiation.cc:240] Negotiation successful
0810 07:18:02.581221 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":100,"threads_started":1}
I20260810 07:18:02.582042 1565 heartbeater.cc:347] Connected to a master server at 127.15.217.126:35977
I20260810 07:18:02.582142 1565 heartbeater.cc:464] Registering TS with master...
I20260810 07:18:02.582370 1565 heartbeater.cc:511] Master 127.15.217.126:35977 requested a full tablet report, sending...
I20260810 07:18:02.582831 1205 ts_manager.cc:194] Registered new tserver with Master: 308b655a3d4d4ba793d2ac7f4e74cd92 (127.15.217.66:36221)
I20260810 07:18:02.583221 1205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.66@KRBTEST.COM'} at 127.15.217.66:38107
W20260810 07:18:02.669741 1568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:02.669994 1568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:02.670022 1568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:02.671336 1568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:02.671394 1568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:02.671423 1568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.67
I20260810 07:18:02.673071 1568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:33041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.67:39441
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.217.67
--webserver_port=35797
--tserver_master_addrs=127.15.217.126:35977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.67
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:02.673379 1568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:02.673641 1568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:02.679009 1576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:02.679015 1573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:02.679147 1574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:02.679232 1568 server_base.cc:1034] running on GCE node
I20260810 07:18:02.679540 1568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:02.679798 1568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:02.680965 1568 hybrid_clock.cc:648] HybridClock initialized: now 1786346282680955 us; error 38 us; skew 500 ppm
Aug 10 07:18:02 dist-test-slave-10pc krb5kdc[569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346282, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:02.683643 1568 init.cc:377] Logged in from keytab as kudu/127.15.217.67@KRBTEST.COM (short username kudu)
I20260810 07:18:02.684165 1568 webserver.cc:460] Webserver started at http://127.15.217.67:35797/ using document root <none> and password file <none>
I20260810 07:18:02.684404 1568 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:02.684510 1568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:02.686044 1568 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:18:02.686715 1583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:02.686888 1568 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:18:02.686983 1568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/wal
uuid: "c55b7534c4324b52a0ef9fb73002d48c"
format_stamp: "Formatted at 2026-08-10 07:17:41 on dist-test-slave-10pc"
server_key: "ef0590cbfab821d2293d13d4770e2089"
server_key_iv: "473ba2ffda0a22d91714f5a63f8a5566"
server_key_version: "encryptionkey@0"
I20260810 07:18:02.687449 1568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:02.705621 1568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:02.707973 1568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:02.708176 1568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:02.708509 1568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:18:02.708900 1568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:18:02.708956 1568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:02.709000 1568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:18:02.709028 1568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:02.715656 1568 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.67:39441
I20260810 07:18:02.715732 1705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.67:39441 every 8 connection(s)
I20260810 07:18:02.716167 1568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
I20260810 07:18:02.717530 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 1568
Aug 10 07:18:02 dist-test-slave-10pc krb5kdc[569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346282, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:18:02.726480 1710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:02.717991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33368 (local address 127.15.217.126:35977)
0810 07:18:02.718118 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:02.718120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:02.718186 (+ 66us) server_negotiation.cc:407] Connection header received
0810 07:18:02.718287 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:02.718288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:02.718302 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:02.718374 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:02.719040 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.719761 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.720833 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.721141 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.721217 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:02.722780 (+ 1563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:02.722782 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:02.722782 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:02.722791 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:02.724855 (+ 2064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:02.725312 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:02.725314 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:02.725314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:02.725382 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:02.725702 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:02.725703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:02.725703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:02.725906 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:02.726120 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:02.726204 (+ 84us) server_negotiation.cc:299] Negotiation successful
0810 07:18:02.726320 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":102,"thread_start_us":65,"threads_started":1}
I20260810 07:18:02.728833 1419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:02.717106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:42363 (local address 127.15.217.126:35977)
0810 07:18:02.717184 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:02.717187 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:02.717573 (+ 386us) server_negotiation.cc:407] Connection header received
0810 07:18:02.718295 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:02.718296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:02.718307 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:02.718374 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:02.719590 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.720263 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.721632 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.721836 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.721920 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:02.725150 (+ 3230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:02.725152 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:02.725152 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:02.725160 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:02.726966 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:02.727626 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:02.727628 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:02.727628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:02.727699 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:02.728007 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:02.728008 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:02.728008 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:02.728148 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:02.728230 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:02.728620 (+ 390us) server_negotiation.cc:299] Negotiation successful
0810 07:18:02.728723 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":38}
I20260810 07:18:02.729094 1708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:02.717165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.67:42363)
0810 07:18:02.717387 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:02.717448 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:02.718168 (+ 720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:02.718465 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:02.718470 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:02.719177 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:18:02.719475 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:02.719483 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.720372 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.720373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:02.721541 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:02.721545 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.721943 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.721945 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:02.722074 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:02.722700 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:18:02.722746 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:18:02.725049 (+ 2303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:18:02.727069 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:02.727080 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:02.727084 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:02.727552 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:02.727777 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:02.727777 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:02.727778 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:02.727920 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:02.728220 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:18:02.728229 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:18:02.728408 (+ 179us) client_negotiation.cc:769] Sending connection context
0810 07:18:02.728579 (+ 171us) client_negotiation.cc:240] Negotiation successful
0810 07:18:02.728789 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":112,"threads_started":1}
I20260810 07:18:02.729650 1706 heartbeater.cc:347] Connected to a master server at 127.15.217.126:35977
I20260810 07:18:02.729768 1706 heartbeater.cc:464] Registering TS with master...
I20260810 07:18:02.730029 1706 heartbeater.cc:511] Master 127.15.217.126:35977 requested a full tablet report, sending...
I20260810 07:18:02.730708 1205 ts_manager.cc:194] Registered new tserver with Master: c55b7534c4324b52a0ef9fb73002d48c (127.15.217.67:39441)
I20260810 07:18:02.731076 1205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.67@KRBTEST.COM'} at 127.15.217.67:42363
I20260810 07:18:02.739228 16229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:18:02.742434 1419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:02.739729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33378 (local address 127.15.217.126:35977)
0810 07:18:02.739812 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:02.739819 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:02.739869 (+ 50us) server_negotiation.cc:407] Connection header received
0810 07:18:02.739967 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:02.739968 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:02.740019 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:02.740091 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:18:02.740233 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.740837 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.741727 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.741927 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.741987 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:02.742170 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:18:02.742228 (+ 58us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0810 07:18:02.742371 (+ 143us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33378: token expired
Metrics: {"server-negotiator.queue_time_us":52}
W20260810 07:18:02.742537 1419 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33378: token expired
W20260810 07:18:02.742700 1128 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.15.217.126:35977)
I20260810 07:18:02.748947 1419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:02.743024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33394 (local address 127.15.217.126:35977)
0810 07:18:02.743094 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:02.743096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:02.743117 (+ 21us) server_negotiation.cc:407] Connection header received
0810 07:18:02.743139 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:02.743140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:02.743149 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:02.743189 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:02.743695 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.744158 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.744911 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.745107 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.745165 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:02.746435 (+ 1270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:02.746436 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:02.746436 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:02.746443 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:02.747867 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:02.748197 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:02.748197 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:02.748198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:02.748253 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:02.748404 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:02.748404 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:02.748404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:02.748591 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:02.748675 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:02.748781 (+ 106us) server_negotiation.cc:299] Negotiation successful
0810 07:18:02.748844 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 07:18:02.750397 1205 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33394:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:18:02.752223 1205 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:18:02.755241 1340 tablet_service.cc:1467] Processing CreateTablet for tablet 1d7ea65e9b7f446f87104a448fb375a9 (DEFAULT_TABLE table=test-table [id=6fc221cc3f7f4084a845531e18e17a34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:18:02.755510 1340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d7ea65e9b7f446f87104a448fb375a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:02.757308 1294 tablet_bootstrap.cc:492] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2: Bootstrap starting.
I20260810 07:18:02.757691 1294 tablet_bootstrap.cc:654] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:02.758742 1294 tablet_bootstrap.cc:492] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2: No bootstrap required, opened a new log
I20260810 07:18:02.758803 1294 ts_tablet_manager.cc:1397] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:18:02.758970 1294 raft_consensus.cc:348] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c55b7534c4324b52a0ef9fb73002d48c" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39441 } } peers { permanent_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 36221 } } peers { permanent_uuid: "7a71dca4901f4ab4b2bc099682fb3ec2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 42967 } }
I20260810 07:18:02.759060 1294 raft_consensus.cc:374] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:02.759112 1294 raft_consensus.cc:729] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a71dca4901f4ab4b2bc099682fb3ec2, State: Initialized, Role: FOLLOWER
I20260810 07:18:02.759172 1294 consensus_queue.cc:260] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c55b7534c4324b52a0ef9fb73002d48c" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39441 } } peers { permanent_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 36221 } } peers { permanent_uuid: "7a71dca4901f4ab4b2bc099682fb3ec2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 42967 } }
I20260810 07:18:02.759277 1294 ts_tablet_manager.cc:1428] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 07:18:02.759336 1424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:02.754783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:39441 (local address 127.0.0.1:54330)
0810 07:18:02.754934 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:02.754948 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:02.754995 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:02.755600 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:02.755602 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:02.755604 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:02.755671 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:02.755673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.756776 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.756777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:02.757981 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:02.757984 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.759046 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.759047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:02.759123 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:02.759151 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 07:18:02.759204 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 07:18:02.759254 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131}
I20260810 07:18:02.759383 1714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:02.755155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54330 (local address 127.15.217.67:39441)
0810 07:18:02.755363 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:02.755368 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:02.755381 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:18:02.755432 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:02.755436 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:02.755454 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:02.755596 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:02.755748 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.756688 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.758090 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.758951 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.759029 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:02.759114 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:02.759226 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 07:18:02.759275 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":133,"threads_started":1}
I20260810 07:18:02.760228 1635 tablet_service.cc:1467] Processing CreateTablet for tablet 1d7ea65e9b7f446f87104a448fb375a9 (DEFAULT_TABLE table=test-table [id=6fc221cc3f7f4084a845531e18e17a34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:18:02.760680 1635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d7ea65e9b7f446f87104a448fb375a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:02.761200 1712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:02.754792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:36221 (local address 127.0.0.1:60718)
0810 07:18:02.755409 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:02.755420 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:02.755474 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:02.757710 (+ 2236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:02.757711 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:02.757713 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:02.757754 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:02.757756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.758927 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.758928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:02.760032 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:02.760035 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.760975 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.760976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:02.761043 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:02.761048 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:18:02.761098 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 07:18:02.761138 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"mutex_wait_us":3,"thread_start_us":93,"threads_started":1}
I20260810 07:18:02.761245 1713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:02.754925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60718 (local address 127.15.217.66:36221)
0810 07:18:02.757421 (+ 2496us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:02.757426 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:02.757439 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:18:02.757489 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:02.757491 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:02.757508 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:02.757593 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:02.757829 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.758826 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.760119 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.760885 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.760965 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:02.761024 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:02.761114 (+ 90us) server_negotiation.cc:299] Negotiation successful
0810 07:18:02.761159 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2450,"thread_start_us":102,"threads_started":1}
I20260810 07:18:02.761929 1495 tablet_service.cc:1467] Processing CreateTablet for tablet 1d7ea65e9b7f446f87104a448fb375a9 (DEFAULT_TABLE table=test-table [id=6fc221cc3f7f4084a845531e18e17a34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:18:02.762339 1495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d7ea65e9b7f446f87104a448fb375a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:02.764058 1716 tablet_bootstrap.cc:492] T 1d7ea65e9b7f446f87104a448fb375a9 P c55b7534c4324b52a0ef9fb73002d48c: Bootstrap starting.
I20260810 07:18:02.764779 1716 tablet_bootstrap.cc:654] T 1d7ea65e9b7f446f87104a448fb375a9 P c55b7534c4324b52a0ef9fb73002d48c: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:02.765333 1716 log.cc:824] T 1d7ea65e9b7f446f87104a448fb375a9 P c55b7534c4324b52a0ef9fb73002d48c: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:02.765861 1718 tablet_bootstrap.cc:492] T 1d7ea65e9b7f446f87104a448fb375a9 P 308b655a3d4d4ba793d2ac7f4e74cd92: Bootstrap starting.
I20260810 07:18:02.766144 1716 tablet_bootstrap.cc:492] T 1d7ea65e9b7f446f87104a448fb375a9 P c55b7534c4324b52a0ef9fb73002d48c: No bootstrap required, opened a new log
I20260810 07:18:02.766229 1716 ts_tablet_manager.cc:1397] T 1d7ea65e9b7f446f87104a448fb375a9 P c55b7534c4324b52a0ef9fb73002d48c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260810 07:18:02.766645 1718 tablet_bootstrap.cc:654] T 1d7ea65e9b7f446f87104a448fb375a9 P 308b655a3d4d4ba793d2ac7f4e74cd92: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:02.767311 1718 log.cc:824] T 1d7ea65e9b7f446f87104a448fb375a9 P 308b655a3d4d4ba793d2ac7f4e74cd92: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:02.768137 1718 tablet_bootstrap.cc:492] T 1d7ea65e9b7f446f87104a448fb375a9 P 308b655a3d4d4ba793d2ac7f4e74cd92: No bootstrap required, opened a new log
I20260810 07:18:02.768230 1718 ts_tablet_manager.cc:1397] T 1d7ea65e9b7f446f87104a448fb375a9 P 308b655a3d4d4ba793d2ac7f4e74cd92: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:18:02.768184 1716 raft_consensus.cc:348] T 1d7ea65e9b7f446f87104a448fb375a9 P c55b7534c4324b52a0ef9fb73002d48c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c55b7534c4324b52a0ef9fb73002d48c" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39441 } } peers { permanent_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 36221 } } peers { permanent_uuid: "7a71dca4901f4ab4b2bc099682fb3ec2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 42967 } }
I20260810 07:18:02.768338 1716 raft_consensus.cc:374] T 1d7ea65e9b7f446f87104a448fb375a9 P c55b7534c4324b52a0ef9fb73002d48c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:02.768391 1716 raft_consensus.cc:729] T 1d7ea65e9b7f446f87104a448fb375a9 P c55b7534c4324b52a0ef9fb73002d48c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c55b7534c4324b52a0ef9fb73002d48c, State: Initialized, Role: FOLLOWER
I20260810 07:18:02.768584 1716 consensus_queue.cc:260] T 1d7ea65e9b7f446f87104a448fb375a9 P c55b7534c4324b52a0ef9fb73002d48c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c55b7534c4324b52a0ef9fb73002d48c" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39441 } } peers { permanent_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 36221 } } peers { permanent_uuid: "7a71dca4901f4ab4b2bc099682fb3ec2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 42967 } }
I20260810 07:18:02.768891 1716 ts_tablet_manager.cc:1428] T 1d7ea65e9b7f446f87104a448fb375a9 P c55b7534c4324b52a0ef9fb73002d48c: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260810 07:18:02.768980 1706 heartbeater.cc:503] Master 127.15.217.126:35977 was elected leader, sending a full tablet report...
I20260810 07:18:02.770475 1718 raft_consensus.cc:348] T 1d7ea65e9b7f446f87104a448fb375a9 P 308b655a3d4d4ba793d2ac7f4e74cd92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c55b7534c4324b52a0ef9fb73002d48c" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39441 } } peers { permanent_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 36221 } } peers { permanent_uuid: "7a71dca4901f4ab4b2bc099682fb3ec2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 42967 } }
I20260810 07:18:02.770648 1718 raft_consensus.cc:374] T 1d7ea65e9b7f446f87104a448fb375a9 P 308b655a3d4d4ba793d2ac7f4e74cd92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:02.770673 1718 raft_consensus.cc:729] T 1d7ea65e9b7f446f87104a448fb375a9 P 308b655a3d4d4ba793d2ac7f4e74cd92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 308b655a3d4d4ba793d2ac7f4e74cd92, State: Initialized, Role: FOLLOWER
I20260810 07:18:02.770767 1718 consensus_queue.cc:260] T 1d7ea65e9b7f446f87104a448fb375a9 P 308b655a3d4d4ba793d2ac7f4e74cd92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c55b7534c4324b52a0ef9fb73002d48c" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39441 } } peers { permanent_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 36221 } } peers { permanent_uuid: "7a71dca4901f4ab4b2bc099682fb3ec2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 42967 } }
I20260810 07:18:02.771019 1718 ts_tablet_manager.cc:1428] T 1d7ea65e9b7f446f87104a448fb375a9 P 308b655a3d4d4ba793d2ac7f4e74cd92: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:18:02.771122 1565 heartbeater.cc:503] Master 127.15.217.126:35977 was elected leader, sending a full tablet report...
W20260810 07:18:02.821671 1566 tablet.cc:2369] T 1d7ea65e9b7f446f87104a448fb375a9 P 308b655a3d4d4ba793d2ac7f4e74cd92: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:18:02.903496 1720 raft_consensus.cc:482] T 1d7ea65e9b7f446f87104a448fb375a9 P c55b7534c4324b52a0ef9fb73002d48c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:18:02.903643 1720 raft_consensus.cc:504] T 1d7ea65e9b7f446f87104a448fb375a9 P c55b7534c4324b52a0ef9fb73002d48c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c55b7534c4324b52a0ef9fb73002d48c" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39441 } } peers { permanent_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 36221 } } peers { permanent_uuid: "7a71dca4901f4ab4b2bc099682fb3ec2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 42967 } }
I20260810 07:18:02.903995 1720 leader_election.cc:288] T 1d7ea65e9b7f446f87104a448fb375a9 P c55b7534c4324b52a0ef9fb73002d48c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 308b655a3d4d4ba793d2ac7f4e74cd92 (127.15.217.66:36221), 7a71dca4901f4ab4b2bc099682fb3ec2 (127.15.217.65:42967)
I20260810 07:18:02.907586 1713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:02.904328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:58643 (local address 127.15.217.66:36221)
0810 07:18:02.904384 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:02.904387 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:02.904443 (+ 56us) server_negotiation.cc:407] Connection header received
0810 07:18:02.904542 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:02.904544 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:02.904557 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:02.904637 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:02.904861 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.905501 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.906634 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.907137 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.907204 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:02.907213 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:02.907439 (+ 226us) server_negotiation.cc:299] Negotiation successful
0810 07:18:02.907491 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 07:18:02.907577 1708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:02.904264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:36221 (local address 127.15.217.67:58643)
0810 07:18:02.904352 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:02.904367 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:02.904441 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:02.904677 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:02.904679 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:02.904681 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:02.904769 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:02.904771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.905610 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.905612 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:02.906526 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:02.906528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.907233 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.907234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:02.907337 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:02.907343 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 07:18:02.907404 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 07:18:02.907454 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260810 07:18:02.907732 1425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:02.904327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:49011 (local address 127.15.217.65:42967)
0810 07:18:02.904385 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:02.904388 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:02.904668 (+ 280us) server_negotiation.cc:407] Connection header received
0810 07:18:02.904723 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:02.904724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:02.904759 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:02.904835 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:02.905065 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.905695 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.906761 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.907269 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.907355 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:02.907464 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:02.907581 (+ 117us) server_negotiation.cc:299] Negotiation successful
0810 07:18:02.907645 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 07:18:02.907740 1722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:02.904264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:42967 (local address 127.15.217.67:49011)
0810 07:18:02.904578 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:02.904590 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:02.904657 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:02.904856 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:02.904857 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:02.904859 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:02.904945 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:02.904947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.905791 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.905792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:02.906662 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:02.906665 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.907369 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.907370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:02.907479 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:02.907484 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:18:02.907558 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 07:18:02.907599 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":103,"thread_start_us":139,"threads_started":1}
I20260810 07:18:02.908067 1518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d7ea65e9b7f446f87104a448fb375a9" candidate_uuid: "c55b7534c4324b52a0ef9fb73002d48c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92" is_pre_election: true
I20260810 07:18:02.908106 1365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d7ea65e9b7f446f87104a448fb375a9" candidate_uuid: "c55b7534c4324b52a0ef9fb73002d48c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a71dca4901f4ab4b2bc099682fb3ec2" is_pre_election: true
I20260810 07:18:02.908265 1518 raft_consensus.cc:2454] T 1d7ea65e9b7f446f87104a448fb375a9 P 308b655a3d4d4ba793d2ac7f4e74cd92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c55b7534c4324b52a0ef9fb73002d48c in term 0.
I20260810 07:18:02.908290 1365 raft_consensus.cc:2454] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c55b7534c4324b52a0ef9fb73002d48c in term 0.
I20260810 07:18:02.908574 1587 leader_election.cc:302] T 1d7ea65e9b7f446f87104a448fb375a9 P c55b7534c4324b52a0ef9fb73002d48c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 308b655a3d4d4ba793d2ac7f4e74cd92, 7a71dca4901f4ab4b2bc099682fb3ec2, c55b7534c4324b52a0ef9fb73002d48c; no voters:
I20260810 07:18:02.908736 1720 raft_consensus.cc:2781] T 1d7ea65e9b7f446f87104a448fb375a9 P c55b7534c4324b52a0ef9fb73002d48c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:18:02.908800 1720 raft_consensus.cc:482] T 1d7ea65e9b7f446f87104a448fb375a9 P c55b7534c4324b52a0ef9fb73002d48c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:18:02.908824 1720 raft_consensus.cc:3037] T 1d7ea65e9b7f446f87104a448fb375a9 P c55b7534c4324b52a0ef9fb73002d48c [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:02.909591 1720 raft_consensus.cc:504] T 1d7ea65e9b7f446f87104a448fb375a9 P c55b7534c4324b52a0ef9fb73002d48c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c55b7534c4324b52a0ef9fb73002d48c" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39441 } } peers { permanent_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 36221 } } peers { permanent_uuid: "7a71dca4901f4ab4b2bc099682fb3ec2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 42967 } }
I20260810 07:18:02.909755 1720 leader_election.cc:288] T 1d7ea65e9b7f446f87104a448fb375a9 P c55b7534c4324b52a0ef9fb73002d48c [CANDIDATE]: Term 1 election: Requested vote from peers 308b655a3d4d4ba793d2ac7f4e74cd92 (127.15.217.66:36221), 7a71dca4901f4ab4b2bc099682fb3ec2 (127.15.217.65:42967)
I20260810 07:18:02.909935 1518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d7ea65e9b7f446f87104a448fb375a9" candidate_uuid: "c55b7534c4324b52a0ef9fb73002d48c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92"
I20260810 07:18:02.909956 1365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d7ea65e9b7f446f87104a448fb375a9" candidate_uuid: "c55b7534c4324b52a0ef9fb73002d48c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a71dca4901f4ab4b2bc099682fb3ec2"
I20260810 07:18:02.910037 1518 raft_consensus.cc:3037] T 1d7ea65e9b7f446f87104a448fb375a9 P 308b655a3d4d4ba793d2ac7f4e74cd92 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:02.910091 1365 raft_consensus.cc:3037] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:02.910866 1518 raft_consensus.cc:2454] T 1d7ea65e9b7f446f87104a448fb375a9 P 308b655a3d4d4ba793d2ac7f4e74cd92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c55b7534c4324b52a0ef9fb73002d48c in term 1.
I20260810 07:18:02.911036 1365 raft_consensus.cc:2454] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c55b7534c4324b52a0ef9fb73002d48c in term 1.
I20260810 07:18:02.911088 1586 leader_election.cc:302] T 1d7ea65e9b7f446f87104a448fb375a9 P c55b7534c4324b52a0ef9fb73002d48c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 308b655a3d4d4ba793d2ac7f4e74cd92, c55b7534c4324b52a0ef9fb73002d48c; no voters:
I20260810 07:18:02.911206 1720 raft_consensus.cc:2781] T 1d7ea65e9b7f446f87104a448fb375a9 P c55b7534c4324b52a0ef9fb73002d48c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:18:02.911361 1720 raft_consensus.cc:686] T 1d7ea65e9b7f446f87104a448fb375a9 P c55b7534c4324b52a0ef9fb73002d48c [term 1 LEADER]: Becoming Leader. State: Replica: c55b7534c4324b52a0ef9fb73002d48c, State: Running, Role: LEADER
I20260810 07:18:02.911475 1720 consensus_queue.cc:237] T 1d7ea65e9b7f446f87104a448fb375a9 P c55b7534c4324b52a0ef9fb73002d48c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c55b7534c4324b52a0ef9fb73002d48c" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39441 } } peers { permanent_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 36221 } } peers { permanent_uuid: "7a71dca4901f4ab4b2bc099682fb3ec2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 42967 } }
I20260810 07:18:02.912499 1205 catalog_manager.cc:5495] T 1d7ea65e9b7f446f87104a448fb375a9 P c55b7534c4324b52a0ef9fb73002d48c reported cstate change: term changed from 0 to 1, leader changed from <none> to c55b7534c4324b52a0ef9fb73002d48c (127.15.217.67). New cstate: current_term: 1 leader_uuid: "c55b7534c4324b52a0ef9fb73002d48c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c55b7534c4324b52a0ef9fb73002d48c" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 36221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a71dca4901f4ab4b2bc099682fb3ec2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 42967 } health_report { overall_health: UNKNOWN } } }
W20260810 07:18:02.938608 1417 tablet.cc:2369] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:18:02.942883 1714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:02.940243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54344 (local address 127.15.217.67:39441)
0810 07:18:02.940310 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:02.940313 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:02.940323 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:18:02.940374 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:02.940375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:02.940396 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:02.940494 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:18:02.940682 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.941259 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.942045 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:02.942260 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:02.942320 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:02.942543 (+ 223us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:18:02.942657 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:18:02.942735 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:02.942776 (+ 41us) server_negotiation.cc:299] Negotiation successful
0810 07:18:02.942797 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 07:18:02.945017 1365 raft_consensus.cc:1261] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 1 FOLLOWER]: Refusing update from remote peer c55b7534c4324b52a0ef9fb73002d48c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:18:02.945050 1518 raft_consensus.cc:1261] T 1d7ea65e9b7f446f87104a448fb375a9 P 308b655a3d4d4ba793d2ac7f4e74cd92 [term 1 FOLLOWER]: Refusing update from remote peer c55b7534c4324b52a0ef9fb73002d48c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:18:02.945425 1723 consensus_queue.cc:1035] T 1d7ea65e9b7f446f87104a448fb375a9 P c55b7534c4324b52a0ef9fb73002d48c [LEADER]: Connected to new peer: Peer: permanent_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 36221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:18:02.945578 1720 consensus_queue.cc:1035] T 1d7ea65e9b7f446f87104a448fb375a9 P c55b7534c4324b52a0ef9fb73002d48c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a71dca4901f4ab4b2bc099682fb3ec2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 42967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:18:02.948181 1729 mvcc.cc:205] Tried to move back new op lower bound from 7316874374939746304 to 7316874374806327296. Current Snapshot: MvccSnapshot[applied={T|T < 7316874374939746304}]
I20260810 07:18:02.948213 1728 mvcc.cc:205] Tried to move back new op lower bound from 7316874374939746304 to 7316874374806327296. Current Snapshot: MvccSnapshot[applied={T|T < 7316874374939746304}]
I20260810 07:18:02.948920 1726 mvcc.cc:205] Tried to move back new op lower bound from 7316874374939746304 to 7316874374806327296. Current Snapshot: MvccSnapshot[applied={T|T < 7316874374939746304 or (T in {7316874374939746304})}]
I20260810 07:18:02.950977 1205 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:18:02.951095 1205 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:18:02.951902 1205 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1: Sending DeleteTablet for 3 replicas of tablet 1d7ea65e9b7f446f87104a448fb375a9
I20260810 07:18:02.952332 1340 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d7ea65e9b7f446f87104a448fb375a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:18:02 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:38402
I20260810 07:18:02.952346 1635 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d7ea65e9b7f446f87104a448fb375a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:18:02 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:54330
I20260810 07:18:02.952375 1495 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d7ea65e9b7f446f87104a448fb375a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:18:02 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:60718
I20260810 07:18:02.952440 1426 tablet_replica.cc:330] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2: stopping tablet replica
I20260810 07:18:02.952530 1426 raft_consensus.cc:2229] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:18:02.952586 1426 raft_consensus.cc:2258] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:18:02.952652 1737 tablet_replica.cc:330] T 1d7ea65e9b7f446f87104a448fb375a9 P 308b655a3d4d4ba793d2ac7f4e74cd92: stopping tablet replica
I20260810 07:18:02.952759 1737 raft_consensus.cc:2229] T 1d7ea65e9b7f446f87104a448fb375a9 P 308b655a3d4d4ba793d2ac7f4e74cd92 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:18:02.952775 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 1247
I20260810 07:18:02.952870 1737 raft_consensus.cc:2258] T 1d7ea65e9b7f446f87104a448fb375a9 P 308b655a3d4d4ba793d2ac7f4e74cd92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:18:02.956903 1737 ts_tablet_manager.cc:1905] T 1d7ea65e9b7f446f87104a448fb375a9 P 308b655a3d4d4ba793d2ac7f4e74cd92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:18:02.958835 1737 ts_tablet_manager.cc:1918] T 1d7ea65e9b7f446f87104a448fb375a9 P 308b655a3d4d4ba793d2ac7f4e74cd92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:18:02.958913 1737 log.cc:1196] T 1d7ea65e9b7f446f87104a448fb375a9 P 308b655a3d4d4ba793d2ac7f4e74cd92: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/wal/wals/1d7ea65e9b7f446f87104a448fb375a9
I20260810 07:18:02.959224 1737 ts_tablet_manager.cc:1939] T 1d7ea65e9b7f446f87104a448fb375a9 P 308b655a3d4d4ba793d2ac7f4e74cd92: Deleting consensus metadata
I20260810 07:18:02.959760 1191 catalog_manager.cc:4841] TS 308b655a3d4d4ba793d2ac7f4e74cd92 (127.15.217.66:36221): tablet 1d7ea65e9b7f446f87104a448fb375a9 (table test-table [id=6fc221cc3f7f4084a845531e18e17a34]) successfully deleted
I20260810 07:18:02.960636 1736 tablet_replica.cc:330] T 1d7ea65e9b7f446f87104a448fb375a9 P c55b7534c4324b52a0ef9fb73002d48c: stopping tablet replica
I20260810 07:18:02.960798 1736 raft_consensus.cc:2229] T 1d7ea65e9b7f446f87104a448fb375a9 P c55b7534c4324b52a0ef9fb73002d48c [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:18:02.960986 1736 raft_consensus.cc:2258] T 1d7ea65e9b7f446f87104a448fb375a9 P c55b7534c4324b52a0ef9fb73002d48c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:18:02.961599 1736 ts_tablet_manager.cc:1905] T 1d7ea65e9b7f446f87104a448fb375a9 P c55b7534c4324b52a0ef9fb73002d48c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:18:02.964221 1736 ts_tablet_manager.cc:1918] T 1d7ea65e9b7f446f87104a448fb375a9 P c55b7534c4324b52a0ef9fb73002d48c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:18:02.964339 1736 log.cc:1196] T 1d7ea65e9b7f446f87104a448fb375a9 P c55b7534c4324b52a0ef9fb73002d48c: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/wal/wals/1d7ea65e9b7f446f87104a448fb375a9
I20260810 07:18:02.964732 1736 ts_tablet_manager.cc:1939] T 1d7ea65e9b7f446f87104a448fb375a9 P c55b7534c4324b52a0ef9fb73002d48c: Deleting consensus metadata
I20260810 07:18:02.965353 1189 catalog_manager.cc:4841] TS c55b7534c4324b52a0ef9fb73002d48c (127.15.217.67:39441): tablet 1d7ea65e9b7f446f87104a448fb375a9 (table test-table [id=6fc221cc3f7f4084a845531e18e17a34]) successfully deleted
W20260810 07:18:02.965876 1192 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.217.65:42967 (error 108)
I20260810 07:18:02.966238 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 1421
I20260810 07:18:02.966440 1712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:02.966274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:42967 (local address 127.0.0.1:38406)
0810 07:18:02.966341 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:02.966393 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.65:42967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260810 07:18:02.969398 1192 catalog_manager.cc:4584] TS 7a71dca4901f4ab4b2bc099682fb3ec2 (127.15.217.65:42967): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1d7ea65e9b7f446f87104a448fb375a9: Network error: Client connection negotiation failed: client connection to 127.15.217.65:42967: connect: Connection refused (error 111)
I20260810 07:18:02.983225 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 1568
I20260810 07:18:02.990808 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 1169
I20260810 07:18:22.999629 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:35977
--webserver_interface=127.15.217.126
--webserver_port=38241
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:33041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:35977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:18:23.089589 1739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:23.089866 1739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:23.089908 1739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:23.091173 1739 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 07:18:23.091243 1739 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 07:18:23.091264 1739 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 07:18:23.091282 1739 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:18:23.091311 1739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:23.091360 1739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:23.091388 1739 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:18:23.091415 1739 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:18:23.093169 1739 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:33041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:35977
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:35977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=38241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:23.093485 1739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:23.093750 1739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:23.099336 1744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:23.099359 1745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:23.099404 1747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:23.099395 1739 server_base.cc:1034] running on GCE node
I20260810 07:18:23.099856 1739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:23.100198 1739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:23.101380 1739 hybrid_clock.cc:648] HybridClock initialized: now 1786346303101370 us; error 52 us; skew 500 ppm
Aug 10 07:18:23 dist-test-slave-10pc krb5kdc[569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346303, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:23.109859 1739 init.cc:377] Logged in from keytab as kudu/127.15.217.126@KRBTEST.COM (short username kudu)
I20260810 07:18:23.110437 1739 webserver.cc:460] Webserver started at http://127.15.217.126:38241/ using document root <none> and password file <none>
I20260810 07:18:23.110734 1739 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:23.110814 1739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:23.112385 1739 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 07:18:23.113214 1755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:23.113427 1739 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:18:23.113525 1739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "1928338c4bf347c1b69f77261844dca1"
format_stamp: "Formatted at 2026-08-10 07:17:41 on dist-test-slave-10pc"
server_key: "59269af0ebc5c8a76e00fa02e1bc9eb8"
server_key_iv: "88f9a2ec389d8b71ba46eb8e610de5d2"
server_key_version: "encryptionkey@0"
I20260810 07:18:23.113965 1739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:23.133797 1739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:23.136238 1739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:23.136368 1739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:23.141076 1739 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:35977
I20260810 07:18:23.141112 1815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:35977 every 8 connection(s)
I20260810 07:18:23.141556 1739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:18:23.143250 1816 sys_catalog.cc:263] Verifying existing consensus state
I20260810 07:18:23.144011 1816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1: Bootstrap starting.
I20260810 07:18:23.146816 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 1739
I20260810 07:18:23.147183 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.65:42967
--local_ip_for_outbound_sockets=127.15.217.65
--tserver_master_addrs=127.15.217.126:35977
--webserver_port=36741
--webserver_interface=127.15.217.65
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:33041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:18:23.154866 1816 log.cc:824] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:23.157505 1816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:18:23.157862 1816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1: Bootstrap complete.
I20260810 07:18:23.160355 1816 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1928338c4bf347c1b69f77261844dca1" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35977 } }
I20260810 07:18:23.160717 1816 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1928338c4bf347c1b69f77261844dca1, State: Initialized, Role: FOLLOWER
I20260810 07:18:23.160955 1816 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1928338c4bf347c1b69f77261844dca1" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35977 } }
I20260810 07:18:23.161063 1816 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:18:23.161126 1816 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:18:23.161177 1816 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [term 2 FOLLOWER]: Advancing to term 3
I20260810 07:18:23.162387 1816 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1928338c4bf347c1b69f77261844dca1" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35977 } }
I20260810 07:18:23.162524 1816 leader_election.cc:302] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1928338c4bf347c1b69f77261844dca1; no voters:
I20260810 07:18:23.162763 1816 leader_election.cc:288] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 07:18:23.162808 1820 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 07:18:23.162940 1820 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [term 3 LEADER]: Becoming Leader. State: Replica: 1928338c4bf347c1b69f77261844dca1, State: Running, Role: LEADER
I20260810 07:18:23.163041 1820 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1928338c4bf347c1b69f77261844dca1" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35977 } }
I20260810 07:18:23.163094 1816 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:18:23.163326 1821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "1928338c4bf347c1b69f77261844dca1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1928338c4bf347c1b69f77261844dca1" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35977 } } }
I20260810 07:18:23.163425 1821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [sys.catalog]: This master's current role is: LEADER
I20260810 07:18:23.163699 1822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1928338c4bf347c1b69f77261844dca1. Latest consensus state: current_term: 3 leader_uuid: "1928338c4bf347c1b69f77261844dca1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1928338c4bf347c1b69f77261844dca1" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 35977 } } }
I20260810 07:18:23.163820 1822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1 [sys.catalog]: This master's current role is: LEADER
I20260810 07:18:23.163986 1824 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:18:23.165027 1824 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:18:23.165138 1824 catalog_manager.cc:1259] Loaded cluster ID: 721586d3450246ceabce0cda93082d71
I20260810 07:18:23.165169 1824 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:18:23.166527 1824 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:18:23.166728 1824 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1: Loaded TSK: 0
I20260810 07:18:23.166775 1824 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1928338c4bf347c1b69f77261844dca1: Loaded TSK: 1
F20260810 07:18:23.167083 1824 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f03f49559ad google::LogMessage::Fail() at ??:0
@ 0x7f03f495888a google::LogMessage::SendToLog() at ??:0
@ 0x7f03f49554de google::LogMessage::Flush() at ??:0
@ 0x7f03f4956a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f03f70c2104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f03f70bfd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f03fc8af1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f03fc8af48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f03fc8af594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f03fc875c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f03fc87fad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f03f68230d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f03f6818954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f03f767c6db start_thread at ??:0
@ 0x7f03f345361f clone at ??:0
W20260810 07:18:23.255111 1818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:23.255380 1818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:23.255424 1818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:23.256753 1818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:23.256821 1818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:23.256872 1818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.65
I20260810 07:18:23.258478 1818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:33041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.65:42967
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.217.65
--webserver_port=36741
--tserver_master_addrs=127.15.217.126:35977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.65
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:23.258798 1818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:23.259080 1818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:23.264627 1860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:23.264779 1818 server_base.cc:1034] running on GCE node
W20260810 07:18:23.264578 1862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:23.264577 1859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:23.265230 1818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:23.265544 1818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:23.266714 1818 hybrid_clock.cc:648] HybridClock initialized: now 1786346303266687 us; error 45 us; skew 500 ppm
Aug 10 07:18:23 dist-test-slave-10pc krb5kdc[569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346303, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:23.269593 1818 init.cc:377] Logged in from keytab as kudu/127.15.217.65@KRBTEST.COM (short username kudu)
I20260810 07:18:23.270201 1818 webserver.cc:460] Webserver started at http://127.15.217.65:36741/ using document root <none> and password file <none>
I20260810 07:18:23.270453 1818 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:23.270503 1818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:23.272006 1818 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:18:23.272804 1869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:23.273101 1818 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:18:23.273216 1818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/wal
uuid: "7a71dca4901f4ab4b2bc099682fb3ec2"
format_stamp: "Formatted at 2026-08-10 07:17:41 on dist-test-slave-10pc"
server_key: "66148bb75b5517b2d9382b42069e2b3e"
server_key_iv: "eb0ea3b6d15a7b8715ac91d65b6a307a"
server_key_version: "encryptionkey@0"
I20260810 07:18:23.273653 1818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:23.287067 1818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:23.289795 1818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:23.289965 1818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:23.290264 1818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:18:23.290766 1876 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 07:18:23.292979 1818 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 07:18:23.293061 1818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260810 07:18:23.293107 1818 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 07:18:23.293802 1818 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 07:18:23.293869 1818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260810 07:18:23.293946 1876 tablet_bootstrap.cc:492] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2: Bootstrap starting.
I20260810 07:18:23.302066 1818 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.65:42967
I20260810 07:18:23.302520 1818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
I20260810 07:18:23.304387 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 1818
I20260810 07:18:23.304804 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.66:36221
--local_ip_for_outbound_sockets=127.15.217.66
--tserver_master_addrs=127.15.217.126:35977
--webserver_port=45335
--webserver_interface=127.15.217.66
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:33041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:18:23.304797 1999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.65:42967 every 8 connection(s)
I20260810 07:18:23.307365 1876 log.cc:824] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:23.309881 1876 tablet_bootstrap.cc:492] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:18:23.310220 1876 tablet_bootstrap.cc:492] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2: Bootstrap complete.
I20260810 07:18:23.311314 1876 ts_tablet_manager.cc:1397] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.008s
I20260810 07:18:23.312500 1876 raft_consensus.cc:348] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c55b7534c4324b52a0ef9fb73002d48c" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39441 } } peers { permanent_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 36221 } } peers { permanent_uuid: "7a71dca4901f4ab4b2bc099682fb3ec2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 42967 } }
I20260810 07:18:23.312817 1876 raft_consensus.cc:729] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a71dca4901f4ab4b2bc099682fb3ec2, State: Initialized, Role: FOLLOWER
I20260810 07:18:23.312988 1876 consensus_queue.cc:260] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c55b7534c4324b52a0ef9fb73002d48c" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39441 } } peers { permanent_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 36221 } } peers { permanent_uuid: "7a71dca4901f4ab4b2bc099682fb3ec2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 42967 } }
I20260810 07:18:23.313328 1876 ts_tablet_manager.cc:1428] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2: Time spent starting tablet: real 0.002s user 0.003s sys 0.002s
W20260810 07:18:23.433789 2004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:23.434036 2004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:23.434079 2004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:23.434209 2003 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:18:23.303396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.65:59939)
0810 07:18:23.303624 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:23.303670 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:23.304366 (+ 696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:23.433968 (+129602us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":107,"threads_started":1}
W20260810 07:18:23.434468 2001 heartbeater.cc:644] Failed to heartbeat to 127.15.217.126:35977 (0 consecutive failures): Network error: Failed to ping master at 127.15.217.126:35977: Client connection negotiation failed: client connection to 127.15.217.126:35977: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 07:18:23.434844 2003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:23.434724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.65:43381)
0810 07:18:23.434783 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:23.434825 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260810 07:18:23.435211 2003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:23.435121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.65:46843)
0810 07:18:23.435167 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:23.435191 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260810 07:18:23.435324 2001 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 07:18:23.435675 2004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:23.435738 2004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:23.435791 2004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.66
I20260810 07:18:23.437355 2004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:33041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.66:36221
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.217.66
--webserver_port=45335
--tserver_master_addrs=127.15.217.126:35977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.66
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:23.437637 2004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:23.437880 2004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:23.443195 2012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:23.443214 2013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:23.443205 2015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:23.443410 2004 server_base.cc:1034] running on GCE node
I20260810 07:18:23.443686 2004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:23.443934 2004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:23.445104 2004 hybrid_clock.cc:648] HybridClock initialized: now 1786346303445084 us; error 35 us; skew 500 ppm
Aug 10 07:18:23 dist-test-slave-10pc krb5kdc[569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346303, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:23.447903 2004 init.cc:377] Logged in from keytab as kudu/127.15.217.66@KRBTEST.COM (short username kudu)
I20260810 07:18:23.448390 2004 webserver.cc:460] Webserver started at http://127.15.217.66:45335/ using document root <none> and password file <none>
I20260810 07:18:23.448660 2004 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:23.448719 2004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:23.450172 2004 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:18:23.450757 2023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:23.450933 2004 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 07:18:23.451081 2004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/wal
uuid: "308b655a3d4d4ba793d2ac7f4e74cd92"
format_stamp: "Formatted at 2026-08-10 07:17:41 on dist-test-slave-10pc"
server_key: "0558f8690cf26aacf559a4a0f1fa565e"
server_key_iv: "cf8e179fb61258221c96cf6df0335485"
server_key_version: "encryptionkey@0"
I20260810 07:18:23.451467 2004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:23.477784 2004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:23.479915 2004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:23.480110 2004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:23.480381 2004 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:18:23.480754 2004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:18:23.480813 2004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:23.480850 2004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:18:23.480899 2004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:23.487317 2004 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.66:36221
I20260810 07:18:23.487401 2148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.66:36221 every 8 connection(s)
I20260810 07:18:23.487802 2004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
I20260810 07:18:23.489060 2151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:23.488654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.66:48655)
0810 07:18:23.488809 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:23.488887 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":79,"threads_started":1}
W20260810 07:18:23.489540 2149 heartbeater.cc:644] Failed to heartbeat to 127.15.217.126:35977 (0 consecutive failures): Network error: Failed to ping master at 127.15.217.126:35977: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
I20260810 07:18:23.489872 2151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:23.489770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.66:52567)
0810 07:18:23.489825 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:23.489853 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 07:18:23.490165 2151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:23.490104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.66:44973)
0810 07:18:23.490129 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:23.490150 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260810 07:18:23.490283 2149 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 07:18:23.493364 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 2004
I20260810 07:18:23.493687 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.67:39441
--local_ip_for_outbound_sockets=127.15.217.67
--tserver_master_addrs=127.15.217.126:35977
--webserver_port=35797
--webserver_interface=127.15.217.67
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:33041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:18:23.580690 2152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:23.580915 2152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:23.580940 2152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:23.582167 2152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:23.582216 2152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:23.582242 2152 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.67
I20260810 07:18:23.583794 2152 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:33041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.67:39441
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.217.67
--webserver_port=35797
--tserver_master_addrs=127.15.217.126:35977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.67
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:23.584095 2152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:23.584370 2152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:23.589809 2158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:23.589870 2160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:23.589809 2157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:23.590176 2152 server_base.cc:1034] running on GCE node
I20260810 07:18:23.590427 2152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:23.590749 2152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:23.591917 2152 hybrid_clock.cc:648] HybridClock initialized: now 1786346303591909 us; error 43 us; skew 500 ppm
Aug 10 07:18:23 dist-test-slave-10pc krb5kdc[569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346303, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:23.594681 2152 init.cc:377] Logged in from keytab as kudu/127.15.217.67@KRBTEST.COM (short username kudu)
I20260810 07:18:23.595172 2152 webserver.cc:460] Webserver started at http://127.15.217.67:35797/ using document root <none> and password file <none>
I20260810 07:18:23.595384 2152 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:23.595427 2152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:23.596822 2152 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:18:23.597561 2167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:23.597797 2152 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:18:23.597901 2152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/wal
uuid: "c55b7534c4324b52a0ef9fb73002d48c"
format_stamp: "Formatted at 2026-08-10 07:17:41 on dist-test-slave-10pc"
server_key: "ef0590cbfab821d2293d13d4770e2089"
server_key_iv: "473ba2ffda0a22d91714f5a63f8a5566"
server_key_version: "encryptionkey@0"
I20260810 07:18:23.598348 2152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:23.620260 2152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:23.623221 2152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:23.623425 2152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:23.623710 2152 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:18:23.624081 2152 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:18:23.624120 2152 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:23.624145 2152 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:18:23.624198 2152 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:23.631068 2152 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.67:39441
I20260810 07:18:23.631181 2288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.67:39441 every 8 connection(s)
I20260810 07:18:23.631562 2152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
I20260810 07:18:23.632835 2293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:23.632351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.67:58701)
0810 07:18:23.632597 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:23.632684 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":134,"threads_started":1}
W20260810 07:18:23.633253 2289 heartbeater.cc:644] Failed to heartbeat to 127.15.217.126:35977 (0 consecutive failures): Network error: Failed to ping master at 127.15.217.126:35977: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
I20260810 07:18:23.633535 2293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:23.633440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.67:53803)
0810 07:18:23.633489 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:23.633517 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 07:18:23.633800 2293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:23.633750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.67:53445)
0810 07:18:23.633770 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:23.633788 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260810 07:18:23.633906 2289 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 07:18:23.640411 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 2152
I20260810 07:18:24.436108 2296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:24.435813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.65:57575)
0810 07:18:24.435983 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:24.436034 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":73,"threads_started":1}
I20260810 07:18:24.467549 2297 raft_consensus.cc:482] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:18:24.467700 2297 raft_consensus.cc:504] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c55b7534c4324b52a0ef9fb73002d48c" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39441 } } peers { permanent_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 36221 } } peers { permanent_uuid: "7a71dca4901f4ab4b2bc099682fb3ec2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 42967 } }
I20260810 07:18:24.468086 2297 leader_election.cc:288] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c55b7534c4324b52a0ef9fb73002d48c (127.15.217.67:39441), 308b655a3d4d4ba793d2ac7f4e74cd92 (127.15.217.66:36221)
Aug 10 07:18:24 dist-test-slave-10pc krb5kdc[569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346303, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for kudu/127.15.217.66@KRBTEST.COM
Aug 10 07:18:24 dist-test-slave-10pc krb5kdc[569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346303, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for kudu/127.15.217.67@KRBTEST.COM
I20260810 07:18:24.480176 2300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:24.468598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:40673 (local address 127.15.217.66:36221)
0810 07:18:24.468776 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:24.468786 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:24.468806 (+ 20us) server_negotiation.cc:407] Connection header received
0810 07:18:24.469255 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:24.469257 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:24.469311 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:24.469477 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:24.470326 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:24.471245 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:24.472648 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:24.473118 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:24.473272 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:24.475922 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:24.475928 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:24.475928 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:24.475942 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:24.478156 (+ 2214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:24.478798 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:24.478802 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:24.478802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:24.478912 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:24.479191 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:24.479192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:24.479192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:24.479478 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:24.479586 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:24.479908 (+ 322us) server_negotiation.cc:299] Negotiation successful
0810 07:18:24.480063 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":89,"threads_started":1}
I20260810 07:18:24.480180 2296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:24.468302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:39441 (local address 127.15.217.65:56501)
0810 07:18:24.468371 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:24.468386 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:24.468489 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:24.469538 (+ 1049us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:24.469542 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:24.470015 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:18:24.470177 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:24.470184 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:24.471439 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:24.471440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:24.472556 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:24.472560 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:24.473246 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:24.473247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:24.473368 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:24.474027 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:18:24.474037 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:18:24.476071 (+ 2034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:18:24.478316 (+ 2245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:24.478317 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:24.478317 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:24.478672 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:24.478982 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:24.478983 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:24.478984 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:24.479113 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:24.479563 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:18:24.479571 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:18:24.479727 (+ 156us) client_negotiation.cc:769] Sending connection context
0810 07:18:24.479847 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 07:18:24.480076 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":46}
I20260810 07:18:24.480198 2299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:24.468437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:56501 (local address 127.15.217.67:39441)
0810 07:18:24.468660 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:24.468676 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:24.468707 (+ 31us) server_negotiation.cc:407] Connection header received
0810 07:18:24.469249 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:24.469252 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:24.469311 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:24.469488 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:24.470324 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:24.471329 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:24.472666 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:24.473150 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:24.473295 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:24.476204 (+ 2909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:24.476209 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:24.476209 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:24.476221 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:24.478195 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:24.478784 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:24.478788 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:24.478789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:24.478903 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:24.479192 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:24.479193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:24.479194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:24.479480 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:24.479590 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:24.479909 (+ 319us) server_negotiation.cc:299] Negotiation successful
0810 07:18:24.480064 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":90,"threads_started":1}
I20260810 07:18:24.480677 2298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:24.468306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:36221 (local address 127.15.217.65:40673)
0810 07:18:24.468477 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:24.468490 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:24.468556 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:24.469538 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:24.469542 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:24.469972 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:18:24.470177 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:24.470184 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:24.471382 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:24.471383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:24.472556 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:24.472560 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:24.473243 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:24.473244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:24.473368 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:24.474051 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:18:24.474056 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:18:24.475744 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:18:24.478264 (+ 2520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:24.478277 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:24.478281 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:24.478674 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:24.479018 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:24.479020 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:24.479020 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:24.479112 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:24.479561 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:18:24.479571 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:18:24.479727 (+ 156us) client_negotiation.cc:769] Sending connection context
0810 07:18:24.479847 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 07:18:24.480167 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":226,"thread_start_us":65,"threads_started":1}
W20260810 07:18:24.481267 1871 leader_election.cc:341] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c55b7534c4324b52a0ef9fb73002d48c (127.15.217.67:39441): Not found: Tablet not found: 1d7ea65e9b7f446f87104a448fb375a9
W20260810 07:18:24.481477 1872 leader_election.cc:341] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 308b655a3d4d4ba793d2ac7f4e74cd92 (127.15.217.66:36221): Not found: Tablet not found: 1d7ea65e9b7f446f87104a448fb375a9
I20260810 07:18:24.481516 1872 leader_election.cc:302] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a71dca4901f4ab4b2bc099682fb3ec2; no voters: 308b655a3d4d4ba793d2ac7f4e74cd92, c55b7534c4324b52a0ef9fb73002d48c
I20260810 07:18:24.481638 2297 raft_consensus.cc:2726] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:18:24.491012 2301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:24.490732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.66:56821)
0810 07:18:24.490880 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:24.490951 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":64,"threads_started":1}
I20260810 07:18:24.634516 2302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:24.634283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.67:60813)
0810 07:18:24.634419 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:24.634472 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":63,"threads_started":1}
I20260810 07:18:25.437040 2303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:25.436760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.65:47675)
0810 07:18:25.436909 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:25.436991 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":77,"threads_started":1}
I20260810 07:18:25.491811 2304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:25.491552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.66:33611)
0810 07:18:25.491716 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:25.491763 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":74,"threads_started":1}
I20260810 07:18:25.635298 2305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:25.635076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.67:40011)
0810 07:18:25.635210 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:25.635250 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":66,"threads_started":1}
I20260810 07:18:26.188823 2306 raft_consensus.cc:482] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:18:26.189009 2306 raft_consensus.cc:504] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c55b7534c4324b52a0ef9fb73002d48c" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39441 } } peers { permanent_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 36221 } } peers { permanent_uuid: "7a71dca4901f4ab4b2bc099682fb3ec2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 42967 } }
I20260810 07:18:26.189287 2306 leader_election.cc:288] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c55b7534c4324b52a0ef9fb73002d48c (127.15.217.67:39441), 308b655a3d4d4ba793d2ac7f4e74cd92 (127.15.217.66:36221)
W20260810 07:18:26.189833 1871 leader_election.cc:341] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c55b7534c4324b52a0ef9fb73002d48c (127.15.217.67:39441): Not found: Tablet not found: 1d7ea65e9b7f446f87104a448fb375a9
W20260810 07:18:26.189916 1872 leader_election.cc:341] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 308b655a3d4d4ba793d2ac7f4e74cd92 (127.15.217.66:36221): Not found: Tablet not found: 1d7ea65e9b7f446f87104a448fb375a9
I20260810 07:18:26.190024 1872 leader_election.cc:302] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a71dca4901f4ab4b2bc099682fb3ec2; no voters: 308b655a3d4d4ba793d2ac7f4e74cd92, c55b7534c4324b52a0ef9fb73002d48c
I20260810 07:18:26.190109 2306 raft_consensus.cc:2726] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:18:26.437969 2307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:26.437697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.65:44123)
0810 07:18:26.437878 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:26.437918 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":118,"threads_started":1}
I20260810 07:18:26.492869 2308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:26.492616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.66:53569)
0810 07:18:26.492767 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:26.492818 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":69,"threads_started":1}
I20260810 07:18:26.636116 2309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:26.635826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.67:43969)
0810 07:18:26.635994 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:26.636040 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":80,"threads_started":1}
I20260810 07:18:27.438840 2310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:27.438565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.65:53931)
0810 07:18:27.438737 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:27.438789 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":93,"threads_started":1}
I20260810 07:18:27.493829 2311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:27.493549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.66:41585)
0810 07:18:27.493724 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:27.493775 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":76,"threads_started":1}
I20260810 07:18:27.637042 2312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:27.636754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.67:38875)
0810 07:18:27.636921 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:27.636971 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":88,"threads_started":1}
I20260810 07:18:28.439849 2313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:28.439583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.65:39273)
0810 07:18:28.439750 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:28.439797 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":71,"threads_started":1}
I20260810 07:18:28.494656 2314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:28.494397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.66:49503)
0810 07:18:28.494559 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:28.494607 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":69,"threads_started":1}
I20260810 07:18:28.637876 2315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:28.637642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.67:43745)
0810 07:18:28.637787 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:28.637827 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":84,"threads_started":1}
I20260810 07:18:28.994874 2316 raft_consensus.cc:482] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:18:28.994982 2316 raft_consensus.cc:504] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c55b7534c4324b52a0ef9fb73002d48c" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39441 } } peers { permanent_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 36221 } } peers { permanent_uuid: "7a71dca4901f4ab4b2bc099682fb3ec2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 42967 } }
I20260810 07:18:28.995177 2316 leader_election.cc:288] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c55b7534c4324b52a0ef9fb73002d48c (127.15.217.67:39441), 308b655a3d4d4ba793d2ac7f4e74cd92 (127.15.217.66:36221)
W20260810 07:18:28.995718 1871 leader_election.cc:341] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c55b7534c4324b52a0ef9fb73002d48c (127.15.217.67:39441): Not found: Tablet not found: 1d7ea65e9b7f446f87104a448fb375a9
W20260810 07:18:28.995834 1872 leader_election.cc:341] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 308b655a3d4d4ba793d2ac7f4e74cd92 (127.15.217.66:36221): Not found: Tablet not found: 1d7ea65e9b7f446f87104a448fb375a9
I20260810 07:18:28.995904 1872 leader_election.cc:302] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a71dca4901f4ab4b2bc099682fb3ec2; no voters: 308b655a3d4d4ba793d2ac7f4e74cd92, c55b7534c4324b52a0ef9fb73002d48c
I20260810 07:18:28.995977 2316 raft_consensus.cc:2726] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:18:29.440729 2317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:29.440427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.65:42115)
0810 07:18:29.440631 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:29.440677 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":125,"threads_started":1}
I20260810 07:18:29.495522 2320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:29.495232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.66:39591)
0810 07:18:29.495397 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:29.495457 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":68,"threads_started":1}
I20260810 07:18:29.638826 2323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:29.638460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.67:37295)
0810 07:18:29.638718 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:29.638769 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":137,"threads_started":1}
I20260810 07:18:30.441708 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:30.441447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.65:46381)
0810 07:18:30.441606 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:30.441656 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":72,"threads_started":1}
I20260810 07:18:30.496400 2325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:30.496110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.66:48835)
0810 07:18:30.496291 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:30.496351 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":82,"threads_started":1}
I20260810 07:18:30.639642 2326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:30.639404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.67:45989)
0810 07:18:30.639545 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:30.639593 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":68,"threads_started":1}
I20260810 07:18:30.916285 2327 raft_consensus.cc:482] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:18:30.916432 2327 raft_consensus.cc:504] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c55b7534c4324b52a0ef9fb73002d48c" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39441 } } peers { permanent_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 36221 } } peers { permanent_uuid: "7a71dca4901f4ab4b2bc099682fb3ec2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 42967 } }
I20260810 07:18:30.916693 2327 leader_election.cc:288] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c55b7534c4324b52a0ef9fb73002d48c (127.15.217.67:39441), 308b655a3d4d4ba793d2ac7f4e74cd92 (127.15.217.66:36221)
W20260810 07:18:30.917145 1872 leader_election.cc:341] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 308b655a3d4d4ba793d2ac7f4e74cd92 (127.15.217.66:36221): Not found: Tablet not found: 1d7ea65e9b7f446f87104a448fb375a9
W20260810 07:18:30.917239 1871 leader_election.cc:341] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c55b7534c4324b52a0ef9fb73002d48c (127.15.217.67:39441): Not found: Tablet not found: 1d7ea65e9b7f446f87104a448fb375a9
I20260810 07:18:30.917311 1871 leader_election.cc:302] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a71dca4901f4ab4b2bc099682fb3ec2; no voters: 308b655a3d4d4ba793d2ac7f4e74cd92, c55b7534c4324b52a0ef9fb73002d48c
I20260810 07:18:30.917424 2327 raft_consensus.cc:2726] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:18:31.442544 2328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:31.442282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.65:43865)
0810 07:18:31.442444 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:31.442492 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":78,"threads_started":1}
I20260810 07:18:31.497254 2329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:31.497002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.66:39057)
0810 07:18:31.497170 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:31.497203 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":90,"threads_started":1}
I20260810 07:18:31.640571 2330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:31.640231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.67:48273)
0810 07:18:31.640395 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:31.640444 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":87,"threads_started":1}
I20260810 07:18:32.443447 2331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:32.443184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.65:41829)
0810 07:18:32.443347 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:32.443396 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":74,"threads_started":1}
I20260810 07:18:32.497970 2332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:32.497737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.66:46981)
0810 07:18:32.497875 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:32.497919 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
I20260810 07:18:32.641340 2333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:32.641090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.67:43723)
0810 07:18:32.641242 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:32.641287 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":69,"threads_started":1}
I20260810 07:18:33.444380 2334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:33.444085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.65:37851)
0810 07:18:33.444276 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:33.444327 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":91,"threads_started":1}
I20260810 07:18:33.498818 2335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:33.498547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.66:51403)
0810 07:18:33.498722 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:33.498770 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":84,"threads_started":1}
I20260810 07:18:33.642213 2337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:33.641945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.67:36231)
0810 07:18:33.642118 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:33.642164 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":72,"threads_started":1}
I20260810 07:18:34.445333 2338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:34.445100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.65:60115)
0810 07:18:34.445243 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:34.445283 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":69,"threads_started":1}
I20260810 07:18:34.499564 2340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:34.499299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.66:60537)
0810 07:18:34.499464 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:34.499510 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":77,"threads_started":1}
I20260810 07:18:34.643044 2341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:34.642754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.67:33557)
0810 07:18:34.642897 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:34.642992 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":70,"threads_started":1}
I20260810 07:18:35.446203 2342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:35.445838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.65:54741)
0810 07:18:35.446096 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:35.446142 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":156,"threads_started":1}
I20260810 07:18:35.500363 2343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:35.500092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.66:59387)
0810 07:18:35.500260 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:35.500308 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":65,"threads_started":1}
I20260810 07:18:35.643925 2345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:35.643671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.67:40705)
0810 07:18:35.643825 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:35.643874 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":64,"threads_started":1}
I20260810 07:18:36.446969 2346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:36.446731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.65:55545)
0810 07:18:36.446871 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:36.446917 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":69,"threads_started":1}
I20260810 07:18:36.501171 2347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:36.500931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.66:45693)
0810 07:18:36.501072 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:36.501120 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":68,"threads_started":1}
I20260810 07:18:36.644843 2348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:36.644612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.67:46331)
0810 07:18:36.644759 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:36.644796 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":80,"threads_started":1}
I20260810 07:18:36.746212 2349 raft_consensus.cc:482] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:18:36.746318 2349 raft_consensus.cc:504] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c55b7534c4324b52a0ef9fb73002d48c" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 39441 } } peers { permanent_uuid: "308b655a3d4d4ba793d2ac7f4e74cd92" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 36221 } } peers { permanent_uuid: "7a71dca4901f4ab4b2bc099682fb3ec2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 42967 } }
I20260810 07:18:36.746544 2349 leader_election.cc:288] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c55b7534c4324b52a0ef9fb73002d48c (127.15.217.67:39441), 308b655a3d4d4ba793d2ac7f4e74cd92 (127.15.217.66:36221)
W20260810 07:18:36.746965 1871 leader_election.cc:341] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c55b7534c4324b52a0ef9fb73002d48c (127.15.217.67:39441): Not found: Tablet not found: 1d7ea65e9b7f446f87104a448fb375a9
W20260810 07:18:36.747046 1872 leader_election.cc:341] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 308b655a3d4d4ba793d2ac7f4e74cd92 (127.15.217.66:36221): Not found: Tablet not found: 1d7ea65e9b7f446f87104a448fb375a9
I20260810 07:18:36.747141 1872 leader_election.cc:302] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a71dca4901f4ab4b2bc099682fb3ec2; no voters: 308b655a3d4d4ba793d2ac7f4e74cd92, c55b7534c4324b52a0ef9fb73002d48c
I20260810 07:18:36.747246 2349 raft_consensus.cc:2726] T 1d7ea65e9b7f446f87104a448fb375a9 P 7a71dca4901f4ab4b2bc099682fb3ec2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:18:37.447814 2350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:37.447564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.65:49465)
0810 07:18:37.447713 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:37.447763 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":75,"threads_started":1}
I20260810 07:18:37.502074 2351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:37.501729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.66:57243)
0810 07:18:37.501922 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:37.502020 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":85,"threads_started":1}
I20260810 07:18:37.645597 2352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:37.645331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.67:43443)
0810 07:18:37.645502 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:37.645547 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":83,"threads_started":1}
I20260810 07:18:38.448915 2353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:38.448651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.65:48541)
0810 07:18:38.448818 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:38.448866 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":66,"threads_started":1}
I20260810 07:18:38.502867 2354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:38.502648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.66:53369)
0810 07:18:38.502775 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:38.502817 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
I20260810 07:18:38.641330 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 1818
I20260810 07:18:38.646538 2355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:38.646282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:35977 (local address 127.15.217.67:57253)
0810 07:18:38.646435 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:38.646490 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.126:35977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":75,"threads_started":1}
I20260810 07:18:38.650422 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 2004
I20260810 07:18:38.657337 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 2152
2026-08-10T07:18:38Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58692 ms)
[----------] 30 tests from SecurityITest (236907 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:18:38 dist-test-slave-10pc krb5kdc[2358](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:18:38 dist-test-slave-10pc krb5kdc[2358](info): set up 2 sockets
Aug 10 07:18:38 dist-test-slave-10pc krb5kdc[2358](info): commencing operation
krb5kdc: starting...
W20260810 07:18:39.770862 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:18:39 dist-test-slave-10pc krb5kdc[2358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:18:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:18:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
I20260810 07:18:39.906911 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:40829
--webserver_interface=127.15.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:40331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:40829
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:18:39.995854 2372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:39.996100 2372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:39.996126 2372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:39.997376 2372 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:18:39.997426 2372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:39.997439 2372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:39.997453 2372 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:18:39.997465 2372 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:18:39.998960 2372 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:40331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:40829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:40829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:39.999187 2372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:39.999415 2372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:18:40.005456 2372 server_base.cc:1034] running on GCE node
W20260810 07:18:40.005414 2381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:40.005403 2379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:40.005414 2378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:40.006055 2372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:40.006383 2372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:40.007617 2372 hybrid_clock.cc:648] HybridClock initialized: now 1786346320007595 us; error 32 us; skew 500 ppm
Aug 10 07:18:40 dist-test-slave-10pc krb5kdc[2358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346320, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:40.010514 2372 init.cc:377] Logged in from keytab as kudu/127.15.217.126@KRBTEST.COM (short username kudu)
I20260810 07:18:40.010957 2372 webserver.cc:460] Webserver started at http://127.15.217.126:42529/ using document root <none> and password file <none>
I20260810 07:18:40.011150 2372 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:40.011194 2372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:40.011286 2372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:40.012372 2372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "f69ceba4433d4224b78cb289acaccde0"
format_stamp: "Formatted at 2026-08-10 07:18:40 on dist-test-slave-10pc"
server_key: "a08acf8ae3edf780e03758e4059ac939"
server_key_iv: "883a2427ef97fa5dadab5f9f88ec4b74"
server_key_version: "encryptionkey@0"
I20260810 07:18:40.012805 2372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "f69ceba4433d4224b78cb289acaccde0"
format_stamp: "Formatted at 2026-08-10 07:18:40 on dist-test-slave-10pc"
server_key: "a08acf8ae3edf780e03758e4059ac939"
server_key_iv: "883a2427ef97fa5dadab5f9f88ec4b74"
server_key_version: "encryptionkey@0"
I20260810 07:18:40.014228 2372 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:18:40.015053 2388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:40.015257 2372 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:18:40.015347 2372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "f69ceba4433d4224b78cb289acaccde0"
format_stamp: "Formatted at 2026-08-10 07:18:40 on dist-test-slave-10pc"
server_key: "a08acf8ae3edf780e03758e4059ac939"
server_key_iv: "883a2427ef97fa5dadab5f9f88ec4b74"
server_key_version: "encryptionkey@0"
I20260810 07:18:40.015407 2372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:40.027345 2372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:40.029848 2372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:40.029992 2372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:40.034152 2372 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:40829
I20260810 07:18:40.034233 2445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:40829 every 8 connection(s)
I20260810 07:18:40.034588 2372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:18:40.035590 2446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:40.038185 2446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f69ceba4433d4224b78cb289acaccde0: Bootstrap starting.
I20260810 07:18:40.038847 2446 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f69ceba4433d4224b78cb289acaccde0: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:40.039165 2446 log.cc:824] T 00000000000000000000000000000000 P f69ceba4433d4224b78cb289acaccde0: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:40.039985 2446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f69ceba4433d4224b78cb289acaccde0: No bootstrap required, opened a new log
I20260810 07:18:40.041851 2446 raft_consensus.cc:348] T 00000000000000000000000000000000 P f69ceba4433d4224b78cb289acaccde0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f69ceba4433d4224b78cb289acaccde0" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 40829 } }
I20260810 07:18:40.042032 2446 raft_consensus.cc:374] T 00000000000000000000000000000000 P f69ceba4433d4224b78cb289acaccde0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:40.042084 2446 raft_consensus.cc:729] T 00000000000000000000000000000000 P f69ceba4433d4224b78cb289acaccde0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f69ceba4433d4224b78cb289acaccde0, State: Initialized, Role: FOLLOWER
I20260810 07:18:40.042258 2446 consensus_queue.cc:260] T 00000000000000000000000000000000 P f69ceba4433d4224b78cb289acaccde0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f69ceba4433d4224b78cb289acaccde0" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 40829 } }
I20260810 07:18:40.042354 2446 raft_consensus.cc:388] T 00000000000000000000000000000000 P f69ceba4433d4224b78cb289acaccde0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:18:40.042394 2446 raft_consensus.cc:482] T 00000000000000000000000000000000 P f69ceba4433d4224b78cb289acaccde0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:18:40.042446 2446 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f69ceba4433d4224b78cb289acaccde0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:40.043074 2446 raft_consensus.cc:504] T 00000000000000000000000000000000 P f69ceba4433d4224b78cb289acaccde0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f69ceba4433d4224b78cb289acaccde0" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 40829 } }
I20260810 07:18:40.043161 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 2372
I20260810 07:18:40.043219 2446 leader_election.cc:302] T 00000000000000000000000000000000 P f69ceba4433d4224b78cb289acaccde0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f69ceba4433d4224b78cb289acaccde0; no voters:
I20260810 07:18:40.043267 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:18:40.043437 2446 leader_election.cc:288] T 00000000000000000000000000000000 P f69ceba4433d4224b78cb289acaccde0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:18:40.043488 16229 external_mini_cluster.cc:1428] Setting key 8aa0e5a0c9c7ddaaca1d72ce2fb0e313
I20260810 07:18:40.043535 2449 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f69ceba4433d4224b78cb289acaccde0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:18:40.043720 2446 sys_catalog.cc:564] T 00000000000000000000000000000000 P f69ceba4433d4224b78cb289acaccde0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:18:40.043758 2449 raft_consensus.cc:686] T 00000000000000000000000000000000 P f69ceba4433d4224b78cb289acaccde0 [term 1 LEADER]: Becoming Leader. State: Replica: f69ceba4433d4224b78cb289acaccde0, State: Running, Role: LEADER
I20260810 07:18:40.044014 2449 consensus_queue.cc:237] T 00000000000000000000000000000000 P f69ceba4433d4224b78cb289acaccde0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f69ceba4433d4224b78cb289acaccde0" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 40829 } }
I20260810 07:18:40.044811 2450 sys_catalog.cc:455] T 00000000000000000000000000000000 P f69ceba4433d4224b78cb289acaccde0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f69ceba4433d4224b78cb289acaccde0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f69ceba4433d4224b78cb289acaccde0" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 40829 } } }
I20260810 07:18:40.044910 2450 sys_catalog.cc:458] T 00000000000000000000000000000000 P f69ceba4433d4224b78cb289acaccde0 [sys.catalog]: This master's current role is: LEADER
I20260810 07:18:40.044860 2451 sys_catalog.cc:455] T 00000000000000000000000000000000 P f69ceba4433d4224b78cb289acaccde0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f69ceba4433d4224b78cb289acaccde0. Latest consensus state: current_term: 1 leader_uuid: "f69ceba4433d4224b78cb289acaccde0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f69ceba4433d4224b78cb289acaccde0" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 40829 } } }
I20260810 07:18:40.044982 2451 sys_catalog.cc:458] T 00000000000000000000000000000000 P f69ceba4433d4224b78cb289acaccde0 [sys.catalog]: This master's current role is: LEADER
I20260810 07:18:40.045374 2460 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:18:40.046000 2460 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:18:40.047912 2460 catalog_manager.cc:1347] Generated new cluster ID: cc8282b809bf4e8188237d647accec59
I20260810 07:18:40.047972 2460 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 07:18:40 dist-test-slave-10pc krb5kdc[2358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:18:40.057041 2453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:40.044247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37034 (local address 127.15.217.126:40829)
0810 07:18:40.044470 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:40.044483 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:40.044524 (+ 41us) server_negotiation.cc:407] Connection header received
0810 07:18:40.044689 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:40.044697 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:40.044766 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:40.045539 (+ 773us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:40.045838 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.047271 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.048394 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.049185 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.049306 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:40.052559 (+ 3253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:40.052565 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:40.052565 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:40.052582 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:40.054966 (+ 2384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:40.055450 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:40.055454 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:40.055454 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:40.055604 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:40.055841 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:40.055842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:40.055843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:40.056239 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:40.056443 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:40.056559 (+ 116us) server_negotiation.cc:299] Negotiation successful
0810 07:18:40.056785 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":78,"threads_started":1}
I20260810 07:18:40.087599 2460 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:18:40.088414 2460 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:18:40.100222 2460 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f69ceba4433d4224b78cb289acaccde0: Generated new TSK 0
I20260810 07:18:40.100521 2460 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab.
Entry for principal kudu/127.15.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab.
I20260810 07:18:40.141101 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.65:0
--local_ip_for_outbound_sockets=127.15.217.65
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:40829
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:40331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:18:40.249095 2474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:40.249334 2474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:40.249365 2474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:40.250695 2474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:40.250736 2474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:40.250767 2474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.65
I20260810 07:18:40.252398 2474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:40331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:40829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.65
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:40.252715 2474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:40.252949 2474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:40.258522 2480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:40.258538 2482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:40.258543 2479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:40.258793 2474 server_base.cc:1034] running on GCE node
I20260810 07:18:40.259150 2474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:40.259445 2474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:40.260624 2474 hybrid_clock.cc:648] HybridClock initialized: now 1786346320260616 us; error 40 us; skew 500 ppm
Aug 10 07:18:40 dist-test-slave-10pc krb5kdc[2358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346320, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:40.263557 2474 init.cc:377] Logged in from keytab as kudu/127.15.217.65@KRBTEST.COM (short username kudu)
I20260810 07:18:40.264094 2474 webserver.cc:460] Webserver started at http://127.15.217.65:41201/ using document root <none> and password file <none>
I20260810 07:18:40.264370 2474 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:40.264444 2474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:40.264659 2474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:40.265875 2474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-0/data/instance:
uuid: "6e8a9509d6f145a0bbbd429e8e6a4ab8"
format_stamp: "Formatted at 2026-08-10 07:18:40 on dist-test-slave-10pc"
server_key: "10678b30f7687b298c7c796eef220c4d"
server_key_iv: "aa00d32664fe23f729838db37c2c14e0"
server_key_version: "encryptionkey@0"
I20260810 07:18:40.266289 2474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance:
uuid: "6e8a9509d6f145a0bbbd429e8e6a4ab8"
format_stamp: "Formatted at 2026-08-10 07:18:40 on dist-test-slave-10pc"
server_key: "10678b30f7687b298c7c796eef220c4d"
server_key_iv: "aa00d32664fe23f729838db37c2c14e0"
server_key_version: "encryptionkey@0"
I20260810 07:18:40.267755 2474 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:18:40.268784 2490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:40.268996 2474 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:18:40.269125 2474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-0/wal
uuid: "6e8a9509d6f145a0bbbd429e8e6a4ab8"
format_stamp: "Formatted at 2026-08-10 07:18:40 on dist-test-slave-10pc"
server_key: "10678b30f7687b298c7c796eef220c4d"
server_key_iv: "aa00d32664fe23f729838db37c2c14e0"
server_key_version: "encryptionkey@0"
I20260810 07:18:40.269212 2474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:40.280176 2474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:40.283031 2474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:40.283236 2474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:40.283515 2474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:18:40.283882 2474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:18:40.283941 2474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:40.283983 2474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:18:40.284008 2474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:40.290572 2474 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.65:38729
I20260810 07:18:40.290627 2616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.65:38729 every 8 connection(s)
I20260810 07:18:40.290973 2474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
Aug 10 07:18:40 dist-test-slave-10pc krb5kdc[2358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346320, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:18:40.298575 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 2474
I20260810 07:18:40.298723 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance
I20260810 07:18:40.298961 16229 external_mini_cluster.cc:1428] Setting key 3a4da11add425103a6565344c5082667
I20260810 07:18:40.301312 2453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:40.291814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:56179 (local address 127.15.217.126:40829)
0810 07:18:40.291876 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:40.291879 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:40.292113 (+ 234us) server_negotiation.cc:407] Connection header received
0810 07:18:40.292724 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:40.292726 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:40.292740 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:40.292806 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:40.293766 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.294335 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.295365 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.295598 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.295697 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:40.298073 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:40.298074 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:40.298075 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:40.298083 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:40.299685 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:40.300205 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:40.300206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:40.300207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:40.300274 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:40.300594 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:40.300595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:40.300596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:40.300739 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:40.300794 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:40.301150 (+ 356us) server_negotiation.cc:299] Negotiation successful
0810 07:18:40.301243 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 07:18:40.301616 2619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:40.291801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:40829 (local address 127.15.217.65:56179)
0810 07:18:40.291971 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:40.292009 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:40.292637 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:40.292865 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:40.292868 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:40.293457 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:18:40.293667 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:40.293672 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.294429 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.294429 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:40.295260 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:40.295263 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.295714 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.295715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:40.295882 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:40.296551 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:18:40.296567 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:18:40.297957 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:18:40.299779 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:40.299787 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:40.299789 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:40.300123 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:40.300346 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:40.300347 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:40.300348 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:40.300520 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:40.300806 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:18:40.300813 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:18:40.300998 (+ 185us) client_negotiation.cc:769] Sending connection context
0810 07:18:40.301109 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 07:18:40.301309 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":99,"threads_started":1}
I20260810 07:18:40.302219 2617 heartbeater.cc:347] Connected to a master server at 127.15.217.126:40829
I20260810 07:18:40.302343 2617 heartbeater.cc:464] Registering TS with master...
I20260810 07:18:40.302584 2617 heartbeater.cc:511] Master 127.15.217.126:40829 requested a full tablet report, sending...
I20260810 07:18:40.303217 2405 ts_manager.cc:194] Registered new tserver with Master: 6e8a9509d6f145a0bbbd429e8e6a4ab8 (127.15.217.65:38729)
I20260810 07:18:40.303905 2405 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.65@KRBTEST.COM'} at 127.15.217.65:56179
WARNING: no policy specified for kudu/127.15.217.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab.
Entry for principal kudu/127.15.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab.
I20260810 07:18:40.332103 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.66:0
--local_ip_for_outbound_sockets=127.15.217.66
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:40829
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:40331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:18:40.420393 2623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:40.420900 2623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:40.420928 2623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:40.422286 2623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:40.422349 2623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:40.422379 2623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.66
I20260810 07:18:40.424058 2623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:40331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:40829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.66
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:40.424381 2623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:40.424705 2623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:40.430430 2631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:40.430436 2629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:40.430457 2628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:40.430812 2623 server_base.cc:1034] running on GCE node
I20260810 07:18:40.431069 2623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:40.431279 2623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:40.432492 2623 hybrid_clock.cc:648] HybridClock initialized: now 1786346320432436 us; error 54 us; skew 500 ppm
Aug 10 07:18:40 dist-test-slave-10pc krb5kdc[2358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346320, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:40.435309 2623 init.cc:377] Logged in from keytab as kudu/127.15.217.66@KRBTEST.COM (short username kudu)
I20260810 07:18:40.435786 2623 webserver.cc:460] Webserver started at http://127.15.217.66:37853/ using document root <none> and password file <none>
I20260810 07:18:40.436012 2623 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:40.436054 2623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:40.436148 2623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:40.437258 2623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-1/data/instance:
uuid: "a02570086b12436d9e6e49b02e7f9628"
format_stamp: "Formatted at 2026-08-10 07:18:40 on dist-test-slave-10pc"
server_key: "d4cbec98488c5258062ffff5608cfec5"
server_key_iv: "bbb01b22c79d980096defd4ab5d313e8"
server_key_version: "encryptionkey@0"
I20260810 07:18:40.437589 2623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance:
uuid: "a02570086b12436d9e6e49b02e7f9628"
format_stamp: "Formatted at 2026-08-10 07:18:40 on dist-test-slave-10pc"
server_key: "d4cbec98488c5258062ffff5608cfec5"
server_key_iv: "bbb01b22c79d980096defd4ab5d313e8"
server_key_version: "encryptionkey@0"
I20260810 07:18:40.439016 2623 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:18:40.439796 2639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:40.440059 2623 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:18:40.440153 2623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-1/wal
uuid: "a02570086b12436d9e6e49b02e7f9628"
format_stamp: "Formatted at 2026-08-10 07:18:40 on dist-test-slave-10pc"
server_key: "d4cbec98488c5258062ffff5608cfec5"
server_key_iv: "bbb01b22c79d980096defd4ab5d313e8"
server_key_version: "encryptionkey@0"
I20260810 07:18:40.440215 2623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:40.457453 2623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:40.459779 2623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:40.459908 2623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:40.460151 2623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:18:40.460441 2623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:18:40.460552 2623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:40.460578 2623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:18:40.460592 2623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:40.467047 2623 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.66:37585
I20260810 07:18:40.467147 2767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.66:37585 every 8 connection(s)
I20260810 07:18:40.467509 2623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
I20260810 07:18:40.468987 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 2623
I20260810 07:18:40.469127 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance
I20260810 07:18:40.469343 16229 external_mini_cluster.cc:1428] Setting key fee1c6b262a678722c05d5df4aa6d4ef
Aug 10 07:18:40 dist-test-slave-10pc krb5kdc[2358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346320, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:18:40.478538 2453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:40.468506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:54217 (local address 127.15.217.126:40829)
0810 07:18:40.468566 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:40.468568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:40.468818 (+ 250us) server_negotiation.cc:407] Connection header received
0810 07:18:40.469488 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:40.469489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:40.469504 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:40.469552 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:40.470587 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.471114 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.472534 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.472744 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.472848 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:40.475319 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:40.475320 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:40.475321 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:40.475330 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:40.476955 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:40.477439 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:40.477440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:40.477440 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:40.477514 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:40.477798 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:40.477799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:40.477799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:40.477930 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:40.478012 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:40.478319 (+ 307us) server_negotiation.cc:299] Negotiation successful
0810 07:18:40.478431 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 07:18:40.478754 2770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:40.468446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:40829 (local address 127.15.217.66:54217)
0810 07:18:40.468662 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:40.468704 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:40.469352 (+ 648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:40.469644 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:40.469647 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:40.470253 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:18:40.470465 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:40.470470 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.471225 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.471227 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:40.472394 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:40.472399 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.472889 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.472890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:40.473076 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:40.473657 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:18:40.473666 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:18:40.475201 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:18:40.477095 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:40.477103 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:40.477106 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:40.477352 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:40.477587 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:40.477588 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:40.477589 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:40.477722 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:40.478007 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:18:40.478015 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:18:40.478172 (+ 157us) client_negotiation.cc:769] Sending connection context
0810 07:18:40.478297 (+ 125us) client_negotiation.cc:240] Negotiation successful
0810 07:18:40.478490 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":99,"threads_started":1}
WARNING: no policy specified for kudu/127.15.217.67@KRBTEST.COM; defaulting to no policy
I20260810 07:18:40.479329 2768 heartbeater.cc:347] Connected to a master server at 127.15.217.126:40829
I20260810 07:18:40.479463 2768 heartbeater.cc:464] Registering TS with master...
I20260810 07:18:40.479663 2768 heartbeater.cc:511] Master 127.15.217.126:40829 requested a full tablet report, sending...
I20260810 07:18:40.480204 2405 ts_manager.cc:194] Registered new tserver with Master: a02570086b12436d9e6e49b02e7f9628 (127.15.217.66:37585)
I20260810 07:18:40.480690 2405 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.66@KRBTEST.COM'} at 127.15.217.66:54217
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab.
Entry for principal kudu/127.15.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab.
I20260810 07:18:40.499326 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.67:0
--local_ip_for_outbound_sockets=127.15.217.67
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:40829
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:40331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:18:40.588752 2773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:40.588987 2773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:40.589011 2773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:40.590137 2773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:40.590175 2773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:40.590201 2773 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.67
I20260810 07:18:40.591643 2773 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:40331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:40829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.67
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:40.591849 2773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:40.592038 2773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:40.597265 2780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:40.597270 2784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:40.597275 2782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:40.597668 2773 server_base.cc:1034] running on GCE node
I20260810 07:18:40.597882 2773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:40.598085 2773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:40.599236 2773 hybrid_clock.cc:648] HybridClock initialized: now 1786346320599224 us; error 31 us; skew 500 ppm
Aug 10 07:18:40 dist-test-slave-10pc krb5kdc[2358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346320, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:40.602085 2773 init.cc:377] Logged in from keytab as kudu/127.15.217.67@KRBTEST.COM (short username kudu)
I20260810 07:18:40.602497 2773 webserver.cc:460] Webserver started at http://127.15.217.67:42771/ using document root <none> and password file <none>
I20260810 07:18:40.602743 2773 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:40.602789 2773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:40.602885 2773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:40.603925 2773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-2/data/instance:
uuid: "d501825e66574e929151d2ee9567d088"
format_stamp: "Formatted at 2026-08-10 07:18:40 on dist-test-slave-10pc"
server_key: "8e6bfa348645bb1f8e6d2faf56806e92"
server_key_iv: "2208a1ba7fc223ab8e7f61f6c8c730af"
server_key_version: "encryptionkey@0"
I20260810 07:18:40.604264 2773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance:
uuid: "d501825e66574e929151d2ee9567d088"
format_stamp: "Formatted at 2026-08-10 07:18:40 on dist-test-slave-10pc"
server_key: "8e6bfa348645bb1f8e6d2faf56806e92"
server_key_iv: "2208a1ba7fc223ab8e7f61f6c8c730af"
server_key_version: "encryptionkey@0"
I20260810 07:18:40.605777 2773 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:18:40.606546 2791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:40.606747 2773 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:18:40.606848 2773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-2/wal
uuid: "d501825e66574e929151d2ee9567d088"
format_stamp: "Formatted at 2026-08-10 07:18:40 on dist-test-slave-10pc"
server_key: "8e6bfa348645bb1f8e6d2faf56806e92"
server_key_iv: "2208a1ba7fc223ab8e7f61f6c8c730af"
server_key_version: "encryptionkey@0"
I20260810 07:18:40.606930 2773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:40.619165 2773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:40.621366 2773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:40.621556 2773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:40.621809 2773 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:18:40.622175 2773 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:18:40.622244 2773 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:40.622291 2773 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:18:40.622318 2773 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:40.628599 2773 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.67:41957
I20260810 07:18:40.628684 2918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.67:41957 every 8 connection(s)
I20260810 07:18:40.629043 2773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
Aug 10 07:18:40 dist-test-slave-10pc krb5kdc[2358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346320, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:18:40.635869 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 2773
I20260810 07:18:40.636001 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance
I20260810 07:18:40.636214 16229 external_mini_cluster.cc:1428] Setting key a441d01eac6f9135a44705857caa44b8
I20260810 07:18:40.639454 2453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:40.629933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:58775 (local address 127.15.217.126:40829)
0810 07:18:40.630043 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:40.630046 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:40.630222 (+ 176us) server_negotiation.cc:407] Connection header received
0810 07:18:40.630826 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:40.630827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:40.630841 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:40.630902 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:40.631934 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.632506 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.633543 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.633758 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.633848 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:40.636169 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:40.636171 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:40.636171 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:40.636179 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:40.637934 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:40.638393 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:40.638394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:40.638395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:40.638467 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:40.638742 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:40.638744 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:40.638744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:40.638880 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:40.638956 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:40.639269 (+ 313us) server_negotiation.cc:299] Negotiation successful
0810 07:18:40.639362 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 07:18:40.639652 2921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:40.629896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:40829 (local address 127.15.217.67:58775)
0810 07:18:40.630087 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:40.630137 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:40.630699 (+ 562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:40.630980 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:40.630985 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:40.631606 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:18:40.631812 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:40.631817 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.632619 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.632620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:40.633432 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:40.633434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.633862 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.633863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:40.633998 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:40.634519 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:18:40.634528 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:18:40.636039 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:18:40.638048 (+ 2009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:40.638056 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:40.638059 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:40.638309 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:40.638545 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:40.638546 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:40.638546 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:40.638665 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:40.638959 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:18:40.638967 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:18:40.639118 (+ 151us) client_negotiation.cc:769] Sending connection context
0810 07:18:40.639231 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 07:18:40.639403 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":85,"threads_started":1}
I20260810 07:18:40.640177 2919 heartbeater.cc:347] Connected to a master server at 127.15.217.126:40829
I20260810 07:18:40.640295 2919 heartbeater.cc:464] Registering TS with master...
I20260810 07:18:40.640556 2919 heartbeater.cc:511] Master 127.15.217.126:40829 requested a full tablet report, sending...
I20260810 07:18:40.641016 2405 ts_manager.cc:194] Registered new tserver with Master: d501825e66574e929151d2ee9567d088 (127.15.217.67:41957)
I20260810 07:18:40.641441 2405 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.67@KRBTEST.COM'} at 127.15.217.67:58775
I20260810 07:18:40.648175 16229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:18:40.655895 2453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:40.649710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37048 (local address 127.15.217.126:40829)
0810 07:18:40.649783 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:40.649785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:40.649902 (+ 117us) server_negotiation.cc:407] Connection header received
0810 07:18:40.649965 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:40.649965 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:40.649974 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:40.650023 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:40.650592 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.651083 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.651823 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.652002 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.652051 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:40.653403 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:40.653404 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:40.653405 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:40.653412 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:40.654845 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:40.655215 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:40.655216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:40.655216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:40.655269 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:40.655452 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:40.655452 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:40.655452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:40.655601 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:40.655676 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:40.655769 (+ 93us) server_negotiation.cc:299] Negotiation successful
0810 07:18:40.655828 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 07:18:40.657698 2405 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37048:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:18:40.659448 2405 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:18:40.667289 2930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:40.662270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:41957 (local address 127.0.0.1:60902)
0810 07:18:40.662589 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:40.662644 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:40.662724 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:40.663087 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:40.663090 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:40.663093 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:40.663174 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:40.663175 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.664643 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.664644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:40.665959 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:40.665962 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.666942 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.666943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:40.667040 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:40.667058 (+ 18us) client_negotiation.cc:769] Sending connection context
0810 07:18:40.667137 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 07:18:40.667198 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":66,"threads_started":1}
I20260810 07:18:40.667732 2934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:40.662465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60902 (local address 127.15.217.67:41957)
0810 07:18:40.662889 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:40.662894 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:40.662905 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:18:40.662948 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:40.662950 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:40.662967 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:40.663262 (+ 295us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:40.663364 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.664528 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.666044 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.666805 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.666915 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:40.666986 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:40.667170 (+ 184us) server_negotiation.cc:299] Negotiation successful
0810 07:18:40.667216 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":120,"threads_started":1}
I20260810 07:18:40.667977 2931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:40.662280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:38729 (local address 127.0.0.1:51022)
0810 07:18:40.662682 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:40.662692 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:40.662735 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:40.663176 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:40.663178 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:40.663181 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:40.663241 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:40.663244 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.666036 (+ 2792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.666036 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:40.666716 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:40.666718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.667711 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.667712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:40.667800 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:40.667805 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:18:40.667861 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 07:18:40.667902 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":89,"threads_started":1}
I20260810 07:18:40.668255 2932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:40.662380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51022 (local address 127.15.217.65:38729)
0810 07:18:40.662913 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:40.662918 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:40.662929 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:18:40.662974 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:40.662977 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:40.662994 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:40.663079 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:40.663335 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.664529 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.666843 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.667644 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.667710 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:40.667772 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:40.668111 (+ 339us) server_negotiation.cc:299] Negotiation successful
0810 07:18:40.668169 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":97,"threads_started":1}
I20260810 07:18:40.668332 2933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:40.662505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:37585 (local address 127.0.0.1:51844)
0810 07:18:40.662839 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:40.662848 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:40.662885 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:40.663205 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:40.663206 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:40.663207 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:40.663241 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:40.663243 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.665847 (+ 2604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.665848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:40.666940 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:40.666943 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.668108 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.668109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:40.668172 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:40.668176 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:18:40.668225 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 07:18:40.668266 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":70,"threads_started":1}
I20260810 07:18:40.668618 2935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:40.662625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51844 (local address 127.15.217.66:37585)
0810 07:18:40.662848 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:40.662854 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:40.662977 (+ 123us) server_negotiation.cc:407] Connection header received
0810 07:18:40.663018 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:40.663022 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:40.663036 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:40.663102 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:40.664562 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.665734 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.667039 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.667941 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.668015 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:40.668070 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:40.668473 (+ 403us) server_negotiation.cc:299] Negotiation successful
0810 07:18:40.668533 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":138,"threads_started":1}
I20260810 07:18:40.668612 2846 tablet_service.cc:1467] Processing CreateTablet for tablet 914246ffbdab4461b0086cf55dcb8695 (DEFAULT_TABLE table=test-table [id=9ad6fb78cd81463487904070801cac73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:18:40.669037 2846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 914246ffbdab4461b0086cf55dcb8695. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:40.669327 2694 tablet_service.cc:1467] Processing CreateTablet for tablet 914246ffbdab4461b0086cf55dcb8695 (DEFAULT_TABLE table=test-table [id=9ad6fb78cd81463487904070801cac73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:18:40.669658 2694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 914246ffbdab4461b0086cf55dcb8695. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:40.672293 2937 tablet_bootstrap.cc:492] T 914246ffbdab4461b0086cf55dcb8695 P d501825e66574e929151d2ee9567d088: Bootstrap starting.
I20260810 07:18:40.673029 2938 tablet_bootstrap.cc:492] T 914246ffbdab4461b0086cf55dcb8695 P a02570086b12436d9e6e49b02e7f9628: Bootstrap starting.
I20260810 07:18:40.673007 2546 tablet_service.cc:1467] Processing CreateTablet for tablet 914246ffbdab4461b0086cf55dcb8695 (DEFAULT_TABLE table=test-table [id=9ad6fb78cd81463487904070801cac73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:18:40.673192 2937 tablet_bootstrap.cc:654] T 914246ffbdab4461b0086cf55dcb8695 P d501825e66574e929151d2ee9567d088: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:40.673377 2546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 914246ffbdab4461b0086cf55dcb8695. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:40.673725 2938 tablet_bootstrap.cc:654] T 914246ffbdab4461b0086cf55dcb8695 P a02570086b12436d9e6e49b02e7f9628: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:40.673985 2937 log.cc:824] T 914246ffbdab4461b0086cf55dcb8695 P d501825e66574e929151d2ee9567d088: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:40.674374 2938 log.cc:824] T 914246ffbdab4461b0086cf55dcb8695 P a02570086b12436d9e6e49b02e7f9628: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:40.674996 2937 tablet_bootstrap.cc:492] T 914246ffbdab4461b0086cf55dcb8695 P d501825e66574e929151d2ee9567d088: No bootstrap required, opened a new log
I20260810 07:18:40.675079 2937 ts_tablet_manager.cc:1397] T 914246ffbdab4461b0086cf55dcb8695 P d501825e66574e929151d2ee9567d088: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:18:40.677030 2941 tablet_bootstrap.cc:492] T 914246ffbdab4461b0086cf55dcb8695 P 6e8a9509d6f145a0bbbd429e8e6a4ab8: Bootstrap starting.
I20260810 07:18:40.677163 2938 tablet_bootstrap.cc:492] T 914246ffbdab4461b0086cf55dcb8695 P a02570086b12436d9e6e49b02e7f9628: No bootstrap required, opened a new log
I20260810 07:18:40.677224 2938 ts_tablet_manager.cc:1397] T 914246ffbdab4461b0086cf55dcb8695 P a02570086b12436d9e6e49b02e7f9628: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260810 07:18:40.677260 2937 raft_consensus.cc:348] T 914246ffbdab4461b0086cf55dcb8695 P d501825e66574e929151d2ee9567d088 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8a9509d6f145a0bbbd429e8e6a4ab8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 38729 } } peers { permanent_uuid: "d501825e66574e929151d2ee9567d088" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 41957 } } peers { permanent_uuid: "a02570086b12436d9e6e49b02e7f9628" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 37585 } }
I20260810 07:18:40.677400 2937 raft_consensus.cc:374] T 914246ffbdab4461b0086cf55dcb8695 P d501825e66574e929151d2ee9567d088 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:40.677420 2937 raft_consensus.cc:729] T 914246ffbdab4461b0086cf55dcb8695 P d501825e66574e929151d2ee9567d088 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d501825e66574e929151d2ee9567d088, State: Initialized, Role: FOLLOWER
I20260810 07:18:40.677560 2937 consensus_queue.cc:260] T 914246ffbdab4461b0086cf55dcb8695 P d501825e66574e929151d2ee9567d088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8a9509d6f145a0bbbd429e8e6a4ab8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 38729 } } peers { permanent_uuid: "d501825e66574e929151d2ee9567d088" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 41957 } } peers { permanent_uuid: "a02570086b12436d9e6e49b02e7f9628" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 37585 } }
I20260810 07:18:40.677742 2941 tablet_bootstrap.cc:654] T 914246ffbdab4461b0086cf55dcb8695 P 6e8a9509d6f145a0bbbd429e8e6a4ab8: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:40.677856 2919 heartbeater.cc:503] Master 127.15.217.126:40829 was elected leader, sending a full tablet report...
I20260810 07:18:40.678103 2937 ts_tablet_manager.cc:1428] T 914246ffbdab4461b0086cf55dcb8695 P d501825e66574e929151d2ee9567d088: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:18:40.678344 2941 log.cc:824] T 914246ffbdab4461b0086cf55dcb8695 P 6e8a9509d6f145a0bbbd429e8e6a4ab8: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:40.679225 2941 tablet_bootstrap.cc:492] T 914246ffbdab4461b0086cf55dcb8695 P 6e8a9509d6f145a0bbbd429e8e6a4ab8: No bootstrap required, opened a new log
I20260810 07:18:40.679301 2941 ts_tablet_manager.cc:1397] T 914246ffbdab4461b0086cf55dcb8695 P 6e8a9509d6f145a0bbbd429e8e6a4ab8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:18:40.679539 2938 raft_consensus.cc:348] T 914246ffbdab4461b0086cf55dcb8695 P a02570086b12436d9e6e49b02e7f9628 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8a9509d6f145a0bbbd429e8e6a4ab8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 38729 } } peers { permanent_uuid: "d501825e66574e929151d2ee9567d088" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 41957 } } peers { permanent_uuid: "a02570086b12436d9e6e49b02e7f9628" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 37585 } }
I20260810 07:18:40.679716 2938 raft_consensus.cc:374] T 914246ffbdab4461b0086cf55dcb8695 P a02570086b12436d9e6e49b02e7f9628 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:40.679767 2938 raft_consensus.cc:729] T 914246ffbdab4461b0086cf55dcb8695 P a02570086b12436d9e6e49b02e7f9628 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a02570086b12436d9e6e49b02e7f9628, State: Initialized, Role: FOLLOWER
I20260810 07:18:40.679896 2938 consensus_queue.cc:260] T 914246ffbdab4461b0086cf55dcb8695 P a02570086b12436d9e6e49b02e7f9628 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8a9509d6f145a0bbbd429e8e6a4ab8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 38729 } } peers { permanent_uuid: "d501825e66574e929151d2ee9567d088" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 41957 } } peers { permanent_uuid: "a02570086b12436d9e6e49b02e7f9628" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 37585 } }
I20260810 07:18:40.680238 2938 ts_tablet_manager.cc:1428] T 914246ffbdab4461b0086cf55dcb8695 P a02570086b12436d9e6e49b02e7f9628: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:18:40.680330 2768 heartbeater.cc:503] Master 127.15.217.126:40829 was elected leader, sending a full tablet report...
I20260810 07:18:40.681314 2941 raft_consensus.cc:348] T 914246ffbdab4461b0086cf55dcb8695 P 6e8a9509d6f145a0bbbd429e8e6a4ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8a9509d6f145a0bbbd429e8e6a4ab8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 38729 } } peers { permanent_uuid: "d501825e66574e929151d2ee9567d088" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 41957 } } peers { permanent_uuid: "a02570086b12436d9e6e49b02e7f9628" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 37585 } }
I20260810 07:18:40.681506 2941 raft_consensus.cc:374] T 914246ffbdab4461b0086cf55dcb8695 P 6e8a9509d6f145a0bbbd429e8e6a4ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:40.681546 2941 raft_consensus.cc:729] T 914246ffbdab4461b0086cf55dcb8695 P 6e8a9509d6f145a0bbbd429e8e6a4ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e8a9509d6f145a0bbbd429e8e6a4ab8, State: Initialized, Role: FOLLOWER
I20260810 07:18:40.681665 2941 consensus_queue.cc:260] T 914246ffbdab4461b0086cf55dcb8695 P 6e8a9509d6f145a0bbbd429e8e6a4ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8a9509d6f145a0bbbd429e8e6a4ab8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 38729 } } peers { permanent_uuid: "d501825e66574e929151d2ee9567d088" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 41957 } } peers { permanent_uuid: "a02570086b12436d9e6e49b02e7f9628" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 37585 } }
I20260810 07:18:40.681964 2941 ts_tablet_manager.cc:1428] T 914246ffbdab4461b0086cf55dcb8695 P 6e8a9509d6f145a0bbbd429e8e6a4ab8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:18:40.681980 2617 heartbeater.cc:503] Master 127.15.217.126:40829 was elected leader, sending a full tablet report...
W20260810 07:18:40.718159 2769 tablet.cc:2369] T 914246ffbdab4461b0086cf55dcb8695 P a02570086b12436d9e6e49b02e7f9628: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:18:40.791850 2618 tablet.cc:2369] T 914246ffbdab4461b0086cf55dcb8695 P 6e8a9509d6f145a0bbbd429e8e6a4ab8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:18:40.879827 2920 tablet.cc:2369] T 914246ffbdab4461b0086cf55dcb8695 P d501825e66574e929151d2ee9567d088: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:18:40.897893 2946 raft_consensus.cc:482] T 914246ffbdab4461b0086cf55dcb8695 P 6e8a9509d6f145a0bbbd429e8e6a4ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:18:40.898087 2946 raft_consensus.cc:504] T 914246ffbdab4461b0086cf55dcb8695 P 6e8a9509d6f145a0bbbd429e8e6a4ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8a9509d6f145a0bbbd429e8e6a4ab8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 38729 } } peers { permanent_uuid: "d501825e66574e929151d2ee9567d088" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 41957 } } peers { permanent_uuid: "a02570086b12436d9e6e49b02e7f9628" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 37585 } }
I20260810 07:18:40.898540 2946 leader_election.cc:288] T 914246ffbdab4461b0086cf55dcb8695 P 6e8a9509d6f145a0bbbd429e8e6a4ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d501825e66574e929151d2ee9567d088 (127.15.217.67:41957), a02570086b12436d9e6e49b02e7f9628 (127.15.217.66:37585)
I20260810 07:18:40.901711 2934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:40.898875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:52427 (local address 127.15.217.67:41957)
0810 07:18:40.898926 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:40.898929 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:40.899005 (+ 76us) server_negotiation.cc:407] Connection header received
0810 07:18:40.899067 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:40.899067 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:40.899078 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:40.899158 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:40.899352 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.899835 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.900931 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.901339 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.901433 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:40.901444 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:40.901594 (+ 150us) server_negotiation.cc:299] Negotiation successful
0810 07:18:40.901639 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 07:18:40.901736 2935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:40.898868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:38513 (local address 127.15.217.66:37585)
0810 07:18:40.898935 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:40.898938 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:40.899049 (+ 111us) server_negotiation.cc:407] Connection header received
0810 07:18:40.899140 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:40.899142 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:40.899156 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:40.899235 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:40.899408 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.899872 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.900923 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.901354 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.901450 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:40.901459 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:40.901622 (+ 163us) server_negotiation.cc:299] Negotiation successful
0810 07:18:40.901665 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 07:18:40.901750 2948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:40.898825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:37585 (local address 127.15.217.65:38513)
0810 07:18:40.898964 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:40.898974 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:40.899060 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:40.899241 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:40.899242 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:40.899244 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:40.899333 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:40.899334 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.899965 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.899967 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:40.900827 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:40.900829 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.901450 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.901451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:40.901533 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:40.901536 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:18:40.901602 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 07:18:40.901647 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":68,"threads_started":1}
I20260810 07:18:40.901764 2947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:40.898744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:41957 (local address 127.15.217.65:52427)
0810 07:18:40.898922 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:40.898935 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:40.899005 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:40.899165 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:40.899166 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:40.899168 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:40.899260 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:40.899262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.899925 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.899926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:40.900830 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:40.900833 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.901435 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.901437 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:40.901512 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:40.901516 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:18:40.901586 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 07:18:40.901630 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":1,"thread_start_us":84,"threads_started":1}
I20260810 07:18:40.902191 2716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "914246ffbdab4461b0086cf55dcb8695" candidate_uuid: "6e8a9509d6f145a0bbbd429e8e6a4ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a02570086b12436d9e6e49b02e7f9628" is_pre_election: true
I20260810 07:18:40.902325 2869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "914246ffbdab4461b0086cf55dcb8695" candidate_uuid: "6e8a9509d6f145a0bbbd429e8e6a4ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d501825e66574e929151d2ee9567d088" is_pre_election: true
I20260810 07:18:40.902392 2716 raft_consensus.cc:2454] T 914246ffbdab4461b0086cf55dcb8695 P a02570086b12436d9e6e49b02e7f9628 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e8a9509d6f145a0bbbd429e8e6a4ab8 in term 0.
I20260810 07:18:40.902465 2869 raft_consensus.cc:2454] T 914246ffbdab4461b0086cf55dcb8695 P d501825e66574e929151d2ee9567d088 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e8a9509d6f145a0bbbd429e8e6a4ab8 in term 0.
I20260810 07:18:40.902688 2492 leader_election.cc:302] T 914246ffbdab4461b0086cf55dcb8695 P 6e8a9509d6f145a0bbbd429e8e6a4ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e8a9509d6f145a0bbbd429e8e6a4ab8, a02570086b12436d9e6e49b02e7f9628; no voters:
I20260810 07:18:40.902854 2946 raft_consensus.cc:2781] T 914246ffbdab4461b0086cf55dcb8695 P 6e8a9509d6f145a0bbbd429e8e6a4ab8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:18:40.902925 2946 raft_consensus.cc:482] T 914246ffbdab4461b0086cf55dcb8695 P 6e8a9509d6f145a0bbbd429e8e6a4ab8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:18:40.902954 2946 raft_consensus.cc:3037] T 914246ffbdab4461b0086cf55dcb8695 P 6e8a9509d6f145a0bbbd429e8e6a4ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:40.903723 2946 raft_consensus.cc:504] T 914246ffbdab4461b0086cf55dcb8695 P 6e8a9509d6f145a0bbbd429e8e6a4ab8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8a9509d6f145a0bbbd429e8e6a4ab8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 38729 } } peers { permanent_uuid: "d501825e66574e929151d2ee9567d088" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 41957 } } peers { permanent_uuid: "a02570086b12436d9e6e49b02e7f9628" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 37585 } }
I20260810 07:18:40.903877 2946 leader_election.cc:288] T 914246ffbdab4461b0086cf55dcb8695 P 6e8a9509d6f145a0bbbd429e8e6a4ab8 [CANDIDATE]: Term 1 election: Requested vote from peers d501825e66574e929151d2ee9567d088 (127.15.217.67:41957), a02570086b12436d9e6e49b02e7f9628 (127.15.217.66:37585)
I20260810 07:18:40.904049 2869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "914246ffbdab4461b0086cf55dcb8695" candidate_uuid: "6e8a9509d6f145a0bbbd429e8e6a4ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d501825e66574e929151d2ee9567d088"
I20260810 07:18:40.904064 2716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "914246ffbdab4461b0086cf55dcb8695" candidate_uuid: "6e8a9509d6f145a0bbbd429e8e6a4ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a02570086b12436d9e6e49b02e7f9628"
I20260810 07:18:40.904150 2869 raft_consensus.cc:3037] T 914246ffbdab4461b0086cf55dcb8695 P d501825e66574e929151d2ee9567d088 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:40.904151 2716 raft_consensus.cc:3037] T 914246ffbdab4461b0086cf55dcb8695 P a02570086b12436d9e6e49b02e7f9628 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:40.905190 2716 raft_consensus.cc:2454] T 914246ffbdab4461b0086cf55dcb8695 P a02570086b12436d9e6e49b02e7f9628 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e8a9509d6f145a0bbbd429e8e6a4ab8 in term 1.
I20260810 07:18:40.905190 2869 raft_consensus.cc:2454] T 914246ffbdab4461b0086cf55dcb8695 P d501825e66574e929151d2ee9567d088 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e8a9509d6f145a0bbbd429e8e6a4ab8 in term 1.
I20260810 07:18:40.905390 2492 leader_election.cc:302] T 914246ffbdab4461b0086cf55dcb8695 P 6e8a9509d6f145a0bbbd429e8e6a4ab8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e8a9509d6f145a0bbbd429e8e6a4ab8, a02570086b12436d9e6e49b02e7f9628; no voters:
I20260810 07:18:40.905516 2946 raft_consensus.cc:2781] T 914246ffbdab4461b0086cf55dcb8695 P 6e8a9509d6f145a0bbbd429e8e6a4ab8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:18:40.905685 2946 raft_consensus.cc:686] T 914246ffbdab4461b0086cf55dcb8695 P 6e8a9509d6f145a0bbbd429e8e6a4ab8 [term 1 LEADER]: Becoming Leader. State: Replica: 6e8a9509d6f145a0bbbd429e8e6a4ab8, State: Running, Role: LEADER
I20260810 07:18:40.905802 2946 consensus_queue.cc:237] T 914246ffbdab4461b0086cf55dcb8695 P 6e8a9509d6f145a0bbbd429e8e6a4ab8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8a9509d6f145a0bbbd429e8e6a4ab8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 38729 } } peers { permanent_uuid: "d501825e66574e929151d2ee9567d088" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 41957 } } peers { permanent_uuid: "a02570086b12436d9e6e49b02e7f9628" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 37585 } }
I20260810 07:18:40.906750 2405 catalog_manager.cc:5495] T 914246ffbdab4461b0086cf55dcb8695 P 6e8a9509d6f145a0bbbd429e8e6a4ab8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e8a9509d6f145a0bbbd429e8e6a4ab8 (127.15.217.65). New cstate: current_term: 1 leader_uuid: "6e8a9509d6f145a0bbbd429e8e6a4ab8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e8a9509d6f145a0bbbd429e8e6a4ab8" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 38729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d501825e66574e929151d2ee9567d088" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 41957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a02570086b12436d9e6e49b02e7f9628" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 37585 } health_report { overall_health: UNKNOWN } } }
I20260810 07:18:40.950424 2932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:40.947751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51038 (local address 127.15.217.65:38729)
0810 07:18:40.947817 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:40.947819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:40.947851 (+ 32us) server_negotiation.cc:407] Connection header received
0810 07:18:40.947894 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:40.947895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:40.947918 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:40.947974 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:18:40.948118 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.948645 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.949531 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:40.949735 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:40.949793 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:40.950033 (+ 240us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:18:40.950157 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:18:40.950240 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:40.950261 (+ 21us) server_negotiation.cc:299] Negotiation successful
0810 07:18:40.950283 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 07:18:40.952646 2869 raft_consensus.cc:1261] T 914246ffbdab4461b0086cf55dcb8695 P d501825e66574e929151d2ee9567d088 [term 1 FOLLOWER]: Refusing update from remote peer 6e8a9509d6f145a0bbbd429e8e6a4ab8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:18:40.952657 2716 raft_consensus.cc:1261] T 914246ffbdab4461b0086cf55dcb8695 P a02570086b12436d9e6e49b02e7f9628 [term 1 FOLLOWER]: Refusing update from remote peer 6e8a9509d6f145a0bbbd429e8e6a4ab8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:18:40.953074 2946 consensus_queue.cc:1035] T 914246ffbdab4461b0086cf55dcb8695 P 6e8a9509d6f145a0bbbd429e8e6a4ab8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a02570086b12436d9e6e49b02e7f9628" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:18:40.953253 2949 consensus_queue.cc:1035] T 914246ffbdab4461b0086cf55dcb8695 P 6e8a9509d6f145a0bbbd429e8e6a4ab8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d501825e66574e929151d2ee9567d088" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 41957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:18:40.955633 2952 mvcc.cc:205] Tried to move back new op lower bound from 7316874530618667008 to 7316874530430967808. Current Snapshot: MvccSnapshot[applied={T|T < 7316874530618667008}]
I20260810 07:18:40.958346 2405 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:18:40.958437 2405 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:18:40.959316 2405 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f69ceba4433d4224b78cb289acaccde0: Sending DeleteTablet for 3 replicas of tablet 914246ffbdab4461b0086cf55dcb8695
I20260810 07:18:40.959718 2546 tablet_service.cc:1514] Processing DeleteTablet for tablet 914246ffbdab4461b0086cf55dcb8695 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:18:40 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:51022
I20260810 07:18:40.959733 2846 tablet_service.cc:1514] Processing DeleteTablet for tablet 914246ffbdab4461b0086cf55dcb8695 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:18:40 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:60902
I20260810 07:18:40.959978 2694 tablet_service.cc:1514] Processing DeleteTablet for tablet 914246ffbdab4461b0086cf55dcb8695 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:18:40 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:51844
I20260810 07:18:40.960101 2966 tablet_replica.cc:330] T 914246ffbdab4461b0086cf55dcb8695 P 6e8a9509d6f145a0bbbd429e8e6a4ab8: stopping tablet replica
I20260810 07:18:40.960110 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 2474
I20260810 07:18:40.960204 2966 raft_consensus.cc:2229] T 914246ffbdab4461b0086cf55dcb8695 P 6e8a9509d6f145a0bbbd429e8e6a4ab8 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:18:40.960373 2967 tablet_replica.cc:330] T 914246ffbdab4461b0086cf55dcb8695 P d501825e66574e929151d2ee9567d088: stopping tablet replica
I20260810 07:18:40.960554 2967 raft_consensus.cc:2229] T 914246ffbdab4461b0086cf55dcb8695 P d501825e66574e929151d2ee9567d088 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:18:40.961417 2967 raft_consensus.cc:2258] T 914246ffbdab4461b0086cf55dcb8695 P d501825e66574e929151d2ee9567d088 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:18:40.962055 2967 ts_tablet_manager.cc:1905] T 914246ffbdab4461b0086cf55dcb8695 P d501825e66574e929151d2ee9567d088: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:18:40.964116 2967 ts_tablet_manager.cc:1918] T 914246ffbdab4461b0086cf55dcb8695 P d501825e66574e929151d2ee9567d088: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:18:40.964221 2967 log.cc:1196] T 914246ffbdab4461b0086cf55dcb8695 P d501825e66574e929151d2ee9567d088: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346081759552-16229-0/minicluster-data/ts-2/wal/wals/914246ffbdab4461b0086cf55dcb8695
I20260810 07:18:40.964557 2967 ts_tablet_manager.cc:1939] T 914246ffbdab4461b0086cf55dcb8695 P d501825e66574e929151d2ee9567d088: Deleting consensus metadata
I20260810 07:18:40.965031 2392 catalog_manager.cc:4841] TS d501825e66574e929151d2ee9567d088 (127.15.217.67:41957): tablet 914246ffbdab4461b0086cf55dcb8695 (table test-table [id=9ad6fb78cd81463487904070801cac73]) successfully deleted
I20260810 07:18:40.968672 2968 tablet_replica.cc:330] T 914246ffbdab4461b0086cf55dcb8695 P a02570086b12436d9e6e49b02e7f9628: stopping tablet replica
I20260810 07:18:40.968950 2968 raft_consensus.cc:2229] T 914246ffbdab4461b0086cf55dcb8695 P a02570086b12436d9e6e49b02e7f9628 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260810 07:18:40.969170 2390 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.217.65:38729 (error 108)
I20260810 07:18:40.969473 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 2623
I20260810 07:18:40.969564 2968 raft_consensus.cc:2258] T 914246ffbdab4461b0086cf55dcb8695 P a02570086b12436d9e6e49b02e7f9628 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:18:40.970602 2933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:40.969637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:38729 (local address 127.0.0.1:51040)
0810 07:18:40.970511 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:40.970561 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.65:38729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":849}
W20260810 07:18:40.970788 2390 catalog_manager.cc:4584] TS 6e8a9509d6f145a0bbbd429e8e6a4ab8 (127.15.217.65:38729): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 914246ffbdab4461b0086cf55dcb8695: Network error: Client connection negotiation failed: client connection to 127.15.217.65:38729: connect: Connection refused (error 111)
I20260810 07:18:40.977818 2933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:40.977670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:37585 (local address 127.0.0.1:51854)
0810 07:18:40.977734 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:40.977772 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.66:37585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 07:18:40.977814 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 2773
I20260810 07:18:40.985819 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 2372
2026-08-10T07:18:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2329 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-10T07:18:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:18:41Z Disabled control of system clock
I20260810 07:18:41.019744 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:41897
--webserver_interface=127.15.217.126
--webserver_port=0
--builtin_ntp_servers=127.15.217.84:45889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:41897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 07:18:41.107194 2973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:41.107460 2973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:41.107493 2973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:41.108704 2973 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:18:41.108748 2973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:41.108762 2973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:41.108774 2973 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:18:41.108788 2973 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:18:41.110152 2973 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:45889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:41897
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:41897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:41.110370 2973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:41.110658 2973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:41.115972 2978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:41.115993 2979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:41.116261 2973 server_base.cc:1034] running on GCE node
W20260810 07:18:41.116325 2981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:41.116622 2973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:41.116969 2973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:41.118145 2973 hybrid_clock.cc:648] HybridClock initialized: now 1786346321118130 us; error 52 us; skew 500 ppm
I20260810 07:18:41.119540 2973 webserver.cc:460] Webserver started at http://127.15.217.126:44675/ using document root <none> and password file <none>
I20260810 07:18:41.119849 2973 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:41.119928 2973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:41.120085 2973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:41.121330 2973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "36135fe3c5ef45728c8fd7dbf6d3e279"
format_stamp: "Formatted at 2026-08-10 07:18:41 on dist-test-slave-10pc"
server_key: "607dad72042b213f113b4a9a00d7da3c"
server_key_iv: "7f08a127d8cb968a0e86947e29189a23"
server_key_version: "encryptionkey@0"
I20260810 07:18:41.121735 2973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "36135fe3c5ef45728c8fd7dbf6d3e279"
format_stamp: "Formatted at 2026-08-10 07:18:41 on dist-test-slave-10pc"
server_key: "607dad72042b213f113b4a9a00d7da3c"
server_key_iv: "7f08a127d8cb968a0e86947e29189a23"
server_key_version: "encryptionkey@0"
I20260810 07:18:41.123178 2973 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 07:18:41.123986 2989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:41.124197 2973 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:18:41.124311 2973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "36135fe3c5ef45728c8fd7dbf6d3e279"
format_stamp: "Formatted at 2026-08-10 07:18:41 on dist-test-slave-10pc"
server_key: "607dad72042b213f113b4a9a00d7da3c"
server_key_iv: "7f08a127d8cb968a0e86947e29189a23"
server_key_version: "encryptionkey@0"
I20260810 07:18:41.124392 2973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:41.139420 2973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:41.139813 2973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:41.139988 2973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:41.144410 2973 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:41897
I20260810 07:18:41.144496 3045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:41897 every 8 connection(s)
I20260810 07:18:41.144942 2973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:18:41.146039 3046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:41.148411 3046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36135fe3c5ef45728c8fd7dbf6d3e279: Bootstrap starting.
I20260810 07:18:41.149071 3046 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36135fe3c5ef45728c8fd7dbf6d3e279: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:41.149327 3046 log.cc:824] T 00000000000000000000000000000000 P 36135fe3c5ef45728c8fd7dbf6d3e279: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:41.150005 3046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36135fe3c5ef45728c8fd7dbf6d3e279: No bootstrap required, opened a new log
I20260810 07:18:41.151644 3046 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36135fe3c5ef45728c8fd7dbf6d3e279 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36135fe3c5ef45728c8fd7dbf6d3e279" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41897 } }
I20260810 07:18:41.151790 3046 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36135fe3c5ef45728c8fd7dbf6d3e279 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:41.151813 3046 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36135fe3c5ef45728c8fd7dbf6d3e279 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36135fe3c5ef45728c8fd7dbf6d3e279, State: Initialized, Role: FOLLOWER
I20260810 07:18:41.151965 3046 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36135fe3c5ef45728c8fd7dbf6d3e279 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36135fe3c5ef45728c8fd7dbf6d3e279" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41897 } }
I20260810 07:18:41.152055 3046 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36135fe3c5ef45728c8fd7dbf6d3e279 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:18:41.152081 3046 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36135fe3c5ef45728c8fd7dbf6d3e279 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:18:41.152108 3046 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36135fe3c5ef45728c8fd7dbf6d3e279 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:41.152694 3046 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36135fe3c5ef45728c8fd7dbf6d3e279 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36135fe3c5ef45728c8fd7dbf6d3e279" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41897 } }
I20260810 07:18:41.152787 3046 leader_election.cc:302] T 00000000000000000000000000000000 P 36135fe3c5ef45728c8fd7dbf6d3e279 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36135fe3c5ef45728c8fd7dbf6d3e279; no voters:
I20260810 07:18:41.152925 3046 leader_election.cc:288] T 00000000000000000000000000000000 P 36135fe3c5ef45728c8fd7dbf6d3e279 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:18:41.153007 3050 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36135fe3c5ef45728c8fd7dbf6d3e279 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:18:41.153205 3050 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36135fe3c5ef45728c8fd7dbf6d3e279 [term 1 LEADER]: Becoming Leader. State: Replica: 36135fe3c5ef45728c8fd7dbf6d3e279, State: Running, Role: LEADER
I20260810 07:18:41.153245 3046 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36135fe3c5ef45728c8fd7dbf6d3e279 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:18:41.153313 3050 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36135fe3c5ef45728c8fd7dbf6d3e279 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36135fe3c5ef45728c8fd7dbf6d3e279" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41897 } }
I20260810 07:18:41.153789 3052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36135fe3c5ef45728c8fd7dbf6d3e279 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36135fe3c5ef45728c8fd7dbf6d3e279. Latest consensus state: current_term: 1 leader_uuid: "36135fe3c5ef45728c8fd7dbf6d3e279" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36135fe3c5ef45728c8fd7dbf6d3e279" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41897 } } }
I20260810 07:18:41.153877 3051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36135fe3c5ef45728c8fd7dbf6d3e279 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36135fe3c5ef45728c8fd7dbf6d3e279" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36135fe3c5ef45728c8fd7dbf6d3e279" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 41897 } } }
I20260810 07:18:41.153980 3052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36135fe3c5ef45728c8fd7dbf6d3e279 [sys.catalog]: This master's current role is: LEADER
I20260810 07:18:41.154047 3051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36135fe3c5ef45728c8fd7dbf6d3e279 [sys.catalog]: This master's current role is: LEADER
I20260810 07:18:41.154408 3064 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:18:41.155005 3064 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:18:41.155121 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 2973
I20260810 07:18:41.155256 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:18:41.155480 16229 external_mini_cluster.cc:1428] Setting key 4a5787582e010b153b1160b02afdf016
I20260810 07:18:41.156908 3064 catalog_manager.cc:1347] Generated new cluster ID: f5f08259aebf4d5fa1a358b8c64de5cc
I20260810 07:18:41.156963 3064 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:18:41.162520 3074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:41.155960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33612 (local address 127.15.217.126:41897)
0810 07:18:41.156584 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:41.156593 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:41.156626 (+ 33us) server_negotiation.cc:407] Connection header received
0810 07:18:41.156754 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:41.156762 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:41.156834 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:41.156975 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:41.157347 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.158731 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.159898 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.160557 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.160681 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:41.161427 (+ 746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:41.161432 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:41.161432 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:18:41.161645 (+ 213us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:41.161686 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:18:41.161700 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:41.161761 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:41.162228 (+ 467us) server_negotiation.cc:299] Negotiation successful
0810 07:18:41.162314 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":88,"threads_started":1}
I20260810 07:18:41.170369 3064 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:18:41.170972 3064 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:18:41.177858 3064 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36135fe3c5ef45728c8fd7dbf6d3e279: Generated new TSK 0
I20260810 07:18:41.178150 3064 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:18:41.183159 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.65:0
--local_ip_for_outbound_sockets=127.15.217.65
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:41897
--builtin_ntp_servers=127.15.217.84:45889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 07:18:41.273236 3076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:41.273463 3076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:41.273485 3076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:41.274695 3076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:41.274741 3076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:41.274767 3076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.65
I20260810 07:18:41.276160 3076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:45889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:41897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.65
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:41.276450 3076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:41.276834 3076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:41.283317 3083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:41.283445 3085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:41.283526 3082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:41.283749 3076 server_base.cc:1034] running on GCE node
I20260810 07:18:41.284006 3076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:41.284272 3076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:41.285455 3076 hybrid_clock.cc:648] HybridClock initialized: now 1786346321285423 us; error 44 us; skew 500 ppm
I20260810 07:18:41.286886 3076 webserver.cc:460] Webserver started at http://127.15.217.65:36365/ using document root <none> and password file <none>
I20260810 07:18:41.287209 3076 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:41.287289 3076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:41.287456 3076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:41.288677 3076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-0/data/instance:
uuid: "63b4199da068459a8b85e793b7cf79d2"
format_stamp: "Formatted at 2026-08-10 07:18:41 on dist-test-slave-10pc"
server_key: "9879955e342e91d649e6223b8c79bff4"
server_key_iv: "58f7a9b2d3c87d3850bb9a82fda5faf2"
server_key_version: "encryptionkey@0"
I20260810 07:18:41.289073 3076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance:
uuid: "63b4199da068459a8b85e793b7cf79d2"
format_stamp: "Formatted at 2026-08-10 07:18:41 on dist-test-slave-10pc"
server_key: "9879955e342e91d649e6223b8c79bff4"
server_key_iv: "58f7a9b2d3c87d3850bb9a82fda5faf2"
server_key_version: "encryptionkey@0"
I20260810 07:18:41.290586 3076 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:18:41.291464 3091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:41.291656 3076 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:18:41.291778 3076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-0/wal
uuid: "63b4199da068459a8b85e793b7cf79d2"
format_stamp: "Formatted at 2026-08-10 07:18:41 on dist-test-slave-10pc"
server_key: "9879955e342e91d649e6223b8c79bff4"
server_key_iv: "58f7a9b2d3c87d3850bb9a82fda5faf2"
server_key_version: "encryptionkey@0"
I20260810 07:18:41.291862 3076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:41.306301 3076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:41.306674 3076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:41.306844 3076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:41.307111 3076 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:18:41.307474 3076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:18:41.307533 3076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:41.307576 3076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:18:41.307616 3076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:41.314436 3076 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.65:40615
I20260810 07:18:41.314517 3218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.65:40615 every 8 connection(s)
I20260810 07:18:41.314929 3076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
I20260810 07:18:41.320385 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 3076
I20260810 07:18:41.320568 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance
I20260810 07:18:41.320786 16229 external_mini_cluster.cc:1428] Setting key b253bf741e04bbfc63cc0811a65395de
I20260810 07:18:41.321309 3074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:41.315836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:52051 (local address 127.15.217.126:41897)
0810 07:18:41.315918 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:41.315935 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:41.316169 (+ 234us) server_negotiation.cc:407] Connection header received
0810 07:18:41.316907 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:41.316909 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:41.316924 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:41.316972 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:41.317766 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.318286 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.319376 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.319602 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.319822 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:41.320950 (+ 1128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:41.320952 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:41.320953 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:18:41.320959 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:41.320978 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:18:41.320983 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:41.321053 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:41.321186 (+ 133us) server_negotiation.cc:299] Negotiation successful
0810 07:18:41.321222 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 07:18:41.321646 3222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:41.315832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41897 (local address 127.15.217.65:52051)
0810 07:18:41.316034 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:41.316071 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:41.316764 (+ 693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:41.317083 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:41.317088 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:41.317103 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:18:41.317657 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:41.317662 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.318405 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.318406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:41.319270 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:41.319273 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.319847 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.319849 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:41.320008 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:41.320777 (+ 769us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:18:41.320778 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:18:41.320809 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:18:41.320835 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:18:41.321080 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:18:41.321088 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:18:41.321092 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:18:41.321176 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 07:18:41.321326 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":75,"threads_started":1}
I20260810 07:18:41.322067 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.66:0
--local_ip_for_outbound_sockets=127.15.217.66
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:41897
--builtin_ntp_servers=127.15.217.84:45889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 07:18:41.322129 3219 heartbeater.cc:347] Connected to a master server at 127.15.217.126:41897
I20260810 07:18:41.322225 3219 heartbeater.cc:464] Registering TS with master...
I20260810 07:18:41.322418 3219 heartbeater.cc:511] Master 127.15.217.126:41897 requested a full tablet report, sending...
I20260810 07:18:41.323024 3007 ts_manager.cc:194] Registered new tserver with Master: 63b4199da068459a8b85e793b7cf79d2 (127.15.217.65:40615)
I20260810 07:18:41.323757 3007 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.217.65:52051
W20260810 07:18:41.415032 3223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:41.415277 3223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:41.415300 3223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:41.416484 3223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:41.416522 3223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:41.416546 3223 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.66
I20260810 07:18:41.417867 3223 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:45889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:41897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.66
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:41.418102 3223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:41.418377 3223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:41.423811 3231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:41.423776 3228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:41.423781 3229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:41.424535 3223 server_base.cc:1034] running on GCE node
I20260810 07:18:41.424821 3223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:41.425108 3223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:41.426290 3223 hybrid_clock.cc:648] HybridClock initialized: now 1786346321426269 us; error 58 us; skew 500 ppm
I20260810 07:18:41.427924 3223 webserver.cc:460] Webserver started at http://127.15.217.66:36013/ using document root <none> and password file <none>
I20260810 07:18:41.428216 3223 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:41.428300 3223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:41.428519 3223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:41.429791 3223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-1/data/instance:
uuid: "71eda82db9ac4c4d9fbebfc7d02464ae"
format_stamp: "Formatted at 2026-08-10 07:18:41 on dist-test-slave-10pc"
server_key: "c0dbbf47b98d1c94a8e788f6759d4954"
server_key_iv: "d66abff2a5e56279b7b49582e815b0c3"
server_key_version: "encryptionkey@0"
I20260810 07:18:41.430212 3223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance:
uuid: "71eda82db9ac4c4d9fbebfc7d02464ae"
format_stamp: "Formatted at 2026-08-10 07:18:41 on dist-test-slave-10pc"
server_key: "c0dbbf47b98d1c94a8e788f6759d4954"
server_key_iv: "d66abff2a5e56279b7b49582e815b0c3"
server_key_version: "encryptionkey@0"
I20260810 07:18:41.431664 3223 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:18:41.432565 3239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:41.432799 3223 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 07:18:41.432909 3223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-1/wal
uuid: "71eda82db9ac4c4d9fbebfc7d02464ae"
format_stamp: "Formatted at 2026-08-10 07:18:41 on dist-test-slave-10pc"
server_key: "c0dbbf47b98d1c94a8e788f6759d4954"
server_key_iv: "d66abff2a5e56279b7b49582e815b0c3"
server_key_version: "encryptionkey@0"
I20260810 07:18:41.433001 3223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:41.451617 3223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:41.451994 3223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:41.452179 3223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:41.452440 3223 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:18:41.452935 3223 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:18:41.453023 3223 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:41.453065 3223 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:18:41.453091 3223 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:41.459908 3223 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.66:39667
I20260810 07:18:41.459951 3366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.66:39667 every 8 connection(s)
I20260810 07:18:41.460439 3223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
I20260810 07:18:41.466522 3074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:41.461386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:43899 (local address 127.15.217.126:41897)
0810 07:18:41.461532 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:41.461535 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:41.461729 (+ 194us) server_negotiation.cc:407] Connection header received
0810 07:18:41.462265 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:41.462266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:41.462433 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:41.462512 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:41.463379 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.463942 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.465084 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.465294 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.465355 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:41.466265 (+ 910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:41.466267 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:41.466267 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:18:41.466272 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:41.466289 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:18:41.466293 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:41.466338 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:41.466427 (+ 89us) server_negotiation.cc:299] Negotiation successful
0810 07:18:41.466456 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260810 07:18:41.466769 3371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:41.461351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41897 (local address 127.15.217.66:43899)
0810 07:18:41.461579 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:41.461624 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:41.462147 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:41.462598 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:41.462602 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:41.462612 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:18:41.463240 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:41.463247 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.464068 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.464069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:41.465006 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:41.465008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.465425 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.465427 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:41.465569 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:41.466116 (+ 547us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:18:41.466117 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:18:41.466151 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:18:41.466177 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:18:41.466355 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:18:41.466360 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:18:41.466362 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 07:18:41.466417 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 07:18:41.466520 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":76,"threads_started":1}
I20260810 07:18:41.467319 3369 heartbeater.cc:347] Connected to a master server at 127.15.217.126:41897
I20260810 07:18:41.467422 3369 heartbeater.cc:464] Registering TS with master...
I20260810 07:18:41.467592 3369 heartbeater.cc:511] Master 127.15.217.126:41897 requested a full tablet report, sending...
I20260810 07:18:41.467974 3007 ts_manager.cc:194] Registered new tserver with Master: 71eda82db9ac4c4d9fbebfc7d02464ae (127.15.217.66:39667)
I20260810 07:18:41.468380 3007 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.217.66:43899
I20260810 07:18:41.469323 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 3223
I20260810 07:18:41.469447 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance
I20260810 07:18:41.469696 16229 external_mini_cluster.cc:1428] Setting key eaf1956d93a736be82cda2dc5fb7637e
I20260810 07:18:41.470842 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.67:0
--local_ip_for_outbound_sockets=127.15.217.67
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:41897
--builtin_ntp_servers=127.15.217.84:45889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 07:18:41.561611 3372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:41.561831 3372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:41.561856 3372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:41.563025 3372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:41.563066 3372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:41.563092 3372 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.67
I20260810 07:18:41.564572 3372 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:45889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:41897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.67
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:41.564800 3372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:41.565120 3372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:41.570595 3378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:41.570585 3380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:41.570582 3377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:41.570837 3372 server_base.cc:1034] running on GCE node
I20260810 07:18:41.571146 3372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:41.571411 3372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:41.572618 3372 hybrid_clock.cc:648] HybridClock initialized: now 1786346321572594 us; error 42 us; skew 500 ppm
I20260810 07:18:41.574106 3372 webserver.cc:460] Webserver started at http://127.15.217.67:39685/ using document root <none> and password file <none>
I20260810 07:18:41.574396 3372 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:41.574476 3372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:41.574656 3372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:41.575801 3372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-2/data/instance:
uuid: "f112dbec3d7a4fa5a3754c7ee3472281"
format_stamp: "Formatted at 2026-08-10 07:18:41 on dist-test-slave-10pc"
server_key: "dcc88fb068c1e7b4b6ea9bc12d93a9bd"
server_key_iv: "4d7172104c0b542cde87f2ae22a48d27"
server_key_version: "encryptionkey@0"
I20260810 07:18:41.576223 3372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance:
uuid: "f112dbec3d7a4fa5a3754c7ee3472281"
format_stamp: "Formatted at 2026-08-10 07:18:41 on dist-test-slave-10pc"
server_key: "dcc88fb068c1e7b4b6ea9bc12d93a9bd"
server_key_iv: "4d7172104c0b542cde87f2ae22a48d27"
server_key_version: "encryptionkey@0"
I20260810 07:18:41.577751 3372 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:18:41.578639 3386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:41.578969 3372 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:18:41.579098 3372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-2/wal
uuid: "f112dbec3d7a4fa5a3754c7ee3472281"
format_stamp: "Formatted at 2026-08-10 07:18:41 on dist-test-slave-10pc"
server_key: "dcc88fb068c1e7b4b6ea9bc12d93a9bd"
server_key_iv: "4d7172104c0b542cde87f2ae22a48d27"
server_key_version: "encryptionkey@0"
I20260810 07:18:41.579205 3372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:41.594784 3372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:41.595151 3372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:41.595335 3372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:41.595608 3372 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:18:41.596063 3372 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:18:41.596122 3372 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:41.596165 3372 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:18:41.596200 3372 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:41.602486 3372 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.67:45561
I20260810 07:18:41.602533 3508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.67:45561 every 8 connection(s)
I20260810 07:18:41.602923 3372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
I20260810 07:18:41.607604 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 3372
I20260810 07:18:41.607734 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance
I20260810 07:18:41.607972 16229 external_mini_cluster.cc:1428] Setting key f6e2a59a42ebcd9e9cc0b1eb07b98397
I20260810 07:18:41.608714 3074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:41.603761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:47545 (local address 127.15.217.126:41897)
0810 07:18:41.603857 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:41.603859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:41.604094 (+ 235us) server_negotiation.cc:407] Connection header received
0810 07:18:41.604646 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:41.604647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:41.604658 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:41.604730 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:41.605475 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.605989 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.607007 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.607210 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.607265 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:41.608299 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:41.608301 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:41.608301 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:18:41.608307 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:41.608327 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:18:41.608331 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:41.608382 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:41.608556 (+ 174us) server_negotiation.cc:299] Negotiation successful
0810 07:18:41.608614 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 07:18:41.608969 3512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:41.603724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:41897 (local address 127.15.217.67:47545)
0810 07:18:41.603944 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:41.603981 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:41.604538 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:41.604818 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:41.604822 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:41.604831 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:18:41.605389 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:41.605393 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.606091 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.606091 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:41.606920 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:41.606922 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.607288 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.607289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:41.607460 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:41.608115 (+ 655us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:18:41.608116 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:18:41.608157 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:18:41.608189 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:18:41.608401 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:18:41.608408 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:18:41.608412 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:18:41.608557 (+ 145us) client_negotiation.cc:240] Negotiation successful
0810 07:18:41.608706 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":109,"threads_started":1}
I20260810 07:18:41.609566 3510 heartbeater.cc:347] Connected to a master server at 127.15.217.126:41897
I20260810 07:18:41.609679 3510 heartbeater.cc:464] Registering TS with master...
I20260810 07:18:41.609887 3510 heartbeater.cc:511] Master 127.15.217.126:41897 requested a full tablet report, sending...
I20260810 07:18:41.610357 3007 ts_manager.cc:194] Registered new tserver with Master: f112dbec3d7a4fa5a3754c7ee3472281 (127.15.217.67:45561)
I20260810 07:18:41.610725 3007 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.217.67:47545
I20260810 07:18:41.620733 16229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:18:41.625286 3074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:41.622257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33626 (local address 127.15.217.126:41897)
0810 07:18:41.622320 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:41.622322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:41.622421 (+ 99us) server_negotiation.cc:407] Connection header received
0810 07:18:41.622467 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:41.622468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:41.622481 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:41.622538 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:41.622729 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.623243 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.624032 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.624245 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.624300 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:41.625038 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:41.625040 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:41.625040 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:18:41.625044 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:41.625061 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:18:41.625065 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:41.625113 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:41.625191 (+ 78us) server_negotiation.cc:299] Negotiation successful
0810 07:18:41.625220 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 07:18:41.627134 3007 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33626:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:18:41.629070 3007 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:18:41.636315 3525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:41.631823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56472 (local address 127.15.217.65:40615)
0810 07:18:41.632052 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:41.632055 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:41.632393 (+ 338us) server_negotiation.cc:407] Connection header received
0810 07:18:41.632494 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:41.632505 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:41.632521 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:41.632604 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:41.632817 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.633916 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.635083 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.635765 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.635843 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:41.635939 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:41.636174 (+ 235us) server_negotiation.cc:299] Negotiation successful
0810 07:18:41.636220 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":103,"threads_started":1}
I20260810 07:18:41.636322 3522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:41.631547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:39667 (local address 127.0.0.1:50934)
0810 07:18:41.632031 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:41.632099 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:41.632165 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:41.632520 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:41.632524 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:41.632526 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:41.632603 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:41.632606 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.634034 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.634035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:41.635231 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:41.635234 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.636086 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.636087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:41.636153 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:41.636157 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:18:41.636204 (+ 47us) client_negotiation.cc:240] Negotiation successful
0810 07:18:41.636234 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"mutex_wait_us":29,"thread_start_us":82,"threads_started":1}
I20260810 07:18:41.636328 3521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:41.631549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:40615 (local address 127.0.0.1:56472)
0810 07:18:41.632310 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:41.632318 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:41.632392 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:41.632650 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:41.632652 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:41.632655 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:41.632728 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:41.632730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.634009 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.634010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:41.634981 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:41.634984 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.635901 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.635902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:41.635981 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:41.635994 (+ 13us) client_negotiation.cc:769] Sending connection context
0810 07:18:41.636151 (+ 157us) client_negotiation.cc:240] Negotiation successful
0810 07:18:41.636220 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":750,"mutex_wait_us":46,"spinlock_wait_cycles":384,"thread_start_us":91,"threads_started":1}
I20260810 07:18:41.636819 3520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:41.631557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:45561 (local address 127.0.0.1:51584)
0810 07:18:41.632503 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:41.632513 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:41.632674 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:41.632910 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:41.632911 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:41.632913 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:41.632946 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:41.632948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.633811 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.633812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:41.635269 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:41.635271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.636321 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.636323 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:41.636624 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:41.636630 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 07:18:41.636684 (+ 54us) client_negotiation.cc:240] Negotiation successful
0810 07:18:41.636735 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":934,"mutex_wait_us":17,"thread_start_us":66,"threads_started":1}
I20260810 07:18:41.637081 3524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:41.631823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50934 (local address 127.15.217.66:39667)
0810 07:18:41.632156 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:41.632160 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:41.632172 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:18:41.632309 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:41.632311 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:41.632328 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:41.632411 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:41.632688 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.633893 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.635338 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.635881 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.635947 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:41.636052 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:41.636946 (+ 894us) server_negotiation.cc:299] Negotiation successful
0810 07:18:41.637003 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":101,"threads_started":1}
I20260810 07:18:41.637169 3523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:41.631643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51584 (local address 127.15.217.67:45561)
0810 07:18:41.631828 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:41.631833 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:41.632752 (+ 919us) server_negotiation.cc:407] Connection header received
0810 07:18:41.632803 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:41.632806 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:41.632822 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:41.632898 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:41.633019 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.633719 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.635353 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.636130 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.636199 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:41.636280 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:41.636994 (+ 714us) server_negotiation.cc:299] Negotiation successful
0810 07:18:41.637077 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":111,"threads_started":1}
I20260810 07:18:41.637286 3144 tablet_service.cc:1467] Processing CreateTablet for tablet ab9e707ab3f344dab9cbee104721a86a (DEFAULT_TABLE table=test-table [id=05e0c027eb1a44ceb5f01095db572a7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:18:41.637719 3144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab9e707ab3f344dab9cbee104721a86a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:41.637746 3292 tablet_service.cc:1467] Processing CreateTablet for tablet ab9e707ab3f344dab9cbee104721a86a (DEFAULT_TABLE table=test-table [id=05e0c027eb1a44ceb5f01095db572a7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:18:41.637857 3437 tablet_service.cc:1467] Processing CreateTablet for tablet ab9e707ab3f344dab9cbee104721a86a (DEFAULT_TABLE table=test-table [id=05e0c027eb1a44ceb5f01095db572a7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:18:41.638160 3292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab9e707ab3f344dab9cbee104721a86a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:41.638206 3437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab9e707ab3f344dab9cbee104721a86a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:41.641191 3526 tablet_bootstrap.cc:492] T ab9e707ab3f344dab9cbee104721a86a P 71eda82db9ac4c4d9fbebfc7d02464ae: Bootstrap starting.
I20260810 07:18:41.641379 3527 tablet_bootstrap.cc:492] T ab9e707ab3f344dab9cbee104721a86a P 63b4199da068459a8b85e793b7cf79d2: Bootstrap starting.
I20260810 07:18:41.641894 3528 tablet_bootstrap.cc:492] T ab9e707ab3f344dab9cbee104721a86a P f112dbec3d7a4fa5a3754c7ee3472281: Bootstrap starting.
I20260810 07:18:41.642071 3527 tablet_bootstrap.cc:654] T ab9e707ab3f344dab9cbee104721a86a P 63b4199da068459a8b85e793b7cf79d2: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:41.642186 3526 tablet_bootstrap.cc:654] T ab9e707ab3f344dab9cbee104721a86a P 71eda82db9ac4c4d9fbebfc7d02464ae: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:41.642464 3527 log.cc:824] T ab9e707ab3f344dab9cbee104721a86a P 63b4199da068459a8b85e793b7cf79d2: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:41.642504 3526 log.cc:824] T ab9e707ab3f344dab9cbee104721a86a P 71eda82db9ac4c4d9fbebfc7d02464ae: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:41.642573 3528 tablet_bootstrap.cc:654] T ab9e707ab3f344dab9cbee104721a86a P f112dbec3d7a4fa5a3754c7ee3472281: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:41.642912 3528 log.cc:824] T ab9e707ab3f344dab9cbee104721a86a P f112dbec3d7a4fa5a3754c7ee3472281: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:41.643462 3527 tablet_bootstrap.cc:492] T ab9e707ab3f344dab9cbee104721a86a P 63b4199da068459a8b85e793b7cf79d2: No bootstrap required, opened a new log
I20260810 07:18:41.643534 3527 ts_tablet_manager.cc:1397] T ab9e707ab3f344dab9cbee104721a86a P 63b4199da068459a8b85e793b7cf79d2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:18:41.643630 3526 tablet_bootstrap.cc:492] T ab9e707ab3f344dab9cbee104721a86a P 71eda82db9ac4c4d9fbebfc7d02464ae: No bootstrap required, opened a new log
I20260810 07:18:41.643707 3526 ts_tablet_manager.cc:1397] T ab9e707ab3f344dab9cbee104721a86a P 71eda82db9ac4c4d9fbebfc7d02464ae: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260810 07:18:41.643916 3528 tablet_bootstrap.cc:492] T ab9e707ab3f344dab9cbee104721a86a P f112dbec3d7a4fa5a3754c7ee3472281: No bootstrap required, opened a new log
I20260810 07:18:41.644035 3528 ts_tablet_manager.cc:1397] T ab9e707ab3f344dab9cbee104721a86a P f112dbec3d7a4fa5a3754c7ee3472281: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:18:41.645479 3527 raft_consensus.cc:348] T ab9e707ab3f344dab9cbee104721a86a P 63b4199da068459a8b85e793b7cf79d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63b4199da068459a8b85e793b7cf79d2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 40615 } } peers { permanent_uuid: "f112dbec3d7a4fa5a3754c7ee3472281" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 45561 } } peers { permanent_uuid: "71eda82db9ac4c4d9fbebfc7d02464ae" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 39667 } }
I20260810 07:18:41.645629 3527 raft_consensus.cc:374] T ab9e707ab3f344dab9cbee104721a86a P 63b4199da068459a8b85e793b7cf79d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:41.645653 3527 raft_consensus.cc:729] T ab9e707ab3f344dab9cbee104721a86a P 63b4199da068459a8b85e793b7cf79d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63b4199da068459a8b85e793b7cf79d2, State: Initialized, Role: FOLLOWER
I20260810 07:18:41.645778 3527 consensus_queue.cc:260] T ab9e707ab3f344dab9cbee104721a86a P 63b4199da068459a8b85e793b7cf79d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63b4199da068459a8b85e793b7cf79d2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 40615 } } peers { permanent_uuid: "f112dbec3d7a4fa5a3754c7ee3472281" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 45561 } } peers { permanent_uuid: "71eda82db9ac4c4d9fbebfc7d02464ae" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 39667 } }
I20260810 07:18:41.646090 3219 heartbeater.cc:503] Master 127.15.217.126:41897 was elected leader, sending a full tablet report...
I20260810 07:18:41.646077 3527 ts_tablet_manager.cc:1428] T ab9e707ab3f344dab9cbee104721a86a P 63b4199da068459a8b85e793b7cf79d2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:18:41.646552 3526 raft_consensus.cc:348] T ab9e707ab3f344dab9cbee104721a86a P 71eda82db9ac4c4d9fbebfc7d02464ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63b4199da068459a8b85e793b7cf79d2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 40615 } } peers { permanent_uuid: "f112dbec3d7a4fa5a3754c7ee3472281" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 45561 } } peers { permanent_uuid: "71eda82db9ac4c4d9fbebfc7d02464ae" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 39667 } }
I20260810 07:18:41.646728 3526 raft_consensus.cc:374] T ab9e707ab3f344dab9cbee104721a86a P 71eda82db9ac4c4d9fbebfc7d02464ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:41.646775 3526 raft_consensus.cc:729] T ab9e707ab3f344dab9cbee104721a86a P 71eda82db9ac4c4d9fbebfc7d02464ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71eda82db9ac4c4d9fbebfc7d02464ae, State: Initialized, Role: FOLLOWER
I20260810 07:18:41.646925 3526 consensus_queue.cc:260] T ab9e707ab3f344dab9cbee104721a86a P 71eda82db9ac4c4d9fbebfc7d02464ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63b4199da068459a8b85e793b7cf79d2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 40615 } } peers { permanent_uuid: "f112dbec3d7a4fa5a3754c7ee3472281" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 45561 } } peers { permanent_uuid: "71eda82db9ac4c4d9fbebfc7d02464ae" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 39667 } }
I20260810 07:18:41.647202 3526 ts_tablet_manager.cc:1428] T ab9e707ab3f344dab9cbee104721a86a P 71eda82db9ac4c4d9fbebfc7d02464ae: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260810 07:18:41.647341 3369 heartbeater.cc:503] Master 127.15.217.126:41897 was elected leader, sending a full tablet report...
I20260810 07:18:41.647737 3528 raft_consensus.cc:348] T ab9e707ab3f344dab9cbee104721a86a P f112dbec3d7a4fa5a3754c7ee3472281 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63b4199da068459a8b85e793b7cf79d2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 40615 } } peers { permanent_uuid: "f112dbec3d7a4fa5a3754c7ee3472281" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 45561 } } peers { permanent_uuid: "71eda82db9ac4c4d9fbebfc7d02464ae" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 39667 } }
I20260810 07:18:41.647876 3528 raft_consensus.cc:374] T ab9e707ab3f344dab9cbee104721a86a P f112dbec3d7a4fa5a3754c7ee3472281 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:41.647939 3528 raft_consensus.cc:729] T ab9e707ab3f344dab9cbee104721a86a P f112dbec3d7a4fa5a3754c7ee3472281 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f112dbec3d7a4fa5a3754c7ee3472281, State: Initialized, Role: FOLLOWER
I20260810 07:18:41.648070 3528 consensus_queue.cc:260] T ab9e707ab3f344dab9cbee104721a86a P f112dbec3d7a4fa5a3754c7ee3472281 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63b4199da068459a8b85e793b7cf79d2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 40615 } } peers { permanent_uuid: "f112dbec3d7a4fa5a3754c7ee3472281" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 45561 } } peers { permanent_uuid: "71eda82db9ac4c4d9fbebfc7d02464ae" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 39667 } }
I20260810 07:18:41.648340 3528 ts_tablet_manager.cc:1428] T ab9e707ab3f344dab9cbee104721a86a P f112dbec3d7a4fa5a3754c7ee3472281: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260810 07:18:41.648406 3510 heartbeater.cc:503] Master 127.15.217.126:41897 was elected leader, sending a full tablet report...
I20260810 07:18:41.666144 3533 raft_consensus.cc:482] T ab9e707ab3f344dab9cbee104721a86a P 63b4199da068459a8b85e793b7cf79d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:18:41.666340 3533 raft_consensus.cc:504] T ab9e707ab3f344dab9cbee104721a86a P 63b4199da068459a8b85e793b7cf79d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63b4199da068459a8b85e793b7cf79d2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 40615 } } peers { permanent_uuid: "f112dbec3d7a4fa5a3754c7ee3472281" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 45561 } } peers { permanent_uuid: "71eda82db9ac4c4d9fbebfc7d02464ae" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 39667 } }
I20260810 07:18:41.666766 3533 leader_election.cc:288] T ab9e707ab3f344dab9cbee104721a86a P 63b4199da068459a8b85e793b7cf79d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f112dbec3d7a4fa5a3754c7ee3472281 (127.15.217.67:45561), 71eda82db9ac4c4d9fbebfc7d02464ae (127.15.217.66:39667)
I20260810 07:18:41.669816 3523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:41.667019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:45989 (local address 127.15.217.67:45561)
0810 07:18:41.667064 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:41.667066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:41.667076 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:18:41.667123 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:41.667124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:41.667135 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:41.667190 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:41.667443 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.667988 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.669062 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.669444 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.669505 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:41.669513 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:41.669687 (+ 174us) server_negotiation.cc:299] Negotiation successful
0810 07:18:41.669739 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 07:18:41.669828 3222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:41.666933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:45561 (local address 127.15.217.65:45989)
0810 07:18:41.666999 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:41.667011 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:41.667059 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:41.667248 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:41.667249 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:41.667250 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:41.667332 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:41.667338 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.668079 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.668080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:41.668971 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:41.668974 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.669530 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.669531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:41.669605 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:41.669608 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:18:41.669660 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 07:18:41.669714 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":17}
I20260810 07:18:41.669898 3524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:41.667067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:55635 (local address 127.15.217.66:39667)
0810 07:18:41.667124 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:41.667127 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:41.667247 (+ 120us) server_negotiation.cc:407] Connection header received
0810 07:18:41.667310 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:41.667311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:41.667323 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:41.667386 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:41.667581 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.668064 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.669110 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.669470 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.669568 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:41.669577 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:41.669770 (+ 193us) server_negotiation.cc:299] Negotiation successful
0810 07:18:41.669819 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 07:18:41.669952 3536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:41.666945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:39667 (local address 127.15.217.65:55635)
0810 07:18:41.667190 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:41.667200 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:41.667247 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:41.667408 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:41.667409 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:41.667411 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:41.667477 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:41.667478 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.668163 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.668164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:41.668993 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:41.668996 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.669594 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.669595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:41.669678 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:41.669683 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:18:41.669742 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 07:18:41.669778 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":104,"threads_started":1}
I20260810 07:18:41.670269 3458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab9e707ab3f344dab9cbee104721a86a" candidate_uuid: "63b4199da068459a8b85e793b7cf79d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f112dbec3d7a4fa5a3754c7ee3472281" is_pre_election: true
I20260810 07:18:41.670269 3312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab9e707ab3f344dab9cbee104721a86a" candidate_uuid: "63b4199da068459a8b85e793b7cf79d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71eda82db9ac4c4d9fbebfc7d02464ae" is_pre_election: true
I20260810 07:18:41.670449 3312 raft_consensus.cc:2454] T ab9e707ab3f344dab9cbee104721a86a P 71eda82db9ac4c4d9fbebfc7d02464ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63b4199da068459a8b85e793b7cf79d2 in term 0.
I20260810 07:18:41.670449 3458 raft_consensus.cc:2454] T ab9e707ab3f344dab9cbee104721a86a P f112dbec3d7a4fa5a3754c7ee3472281 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63b4199da068459a8b85e793b7cf79d2 in term 0.
I20260810 07:18:41.670699 3094 leader_election.cc:302] T ab9e707ab3f344dab9cbee104721a86a P 63b4199da068459a8b85e793b7cf79d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63b4199da068459a8b85e793b7cf79d2, 71eda82db9ac4c4d9fbebfc7d02464ae; no voters:
I20260810 07:18:41.670882 3533 raft_consensus.cc:2781] T ab9e707ab3f344dab9cbee104721a86a P 63b4199da068459a8b85e793b7cf79d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:18:41.670958 3533 raft_consensus.cc:482] T ab9e707ab3f344dab9cbee104721a86a P 63b4199da068459a8b85e793b7cf79d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:18:41.670977 3533 raft_consensus.cc:3037] T ab9e707ab3f344dab9cbee104721a86a P 63b4199da068459a8b85e793b7cf79d2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:41.671772 3533 raft_consensus.cc:504] T ab9e707ab3f344dab9cbee104721a86a P 63b4199da068459a8b85e793b7cf79d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63b4199da068459a8b85e793b7cf79d2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 40615 } } peers { permanent_uuid: "f112dbec3d7a4fa5a3754c7ee3472281" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 45561 } } peers { permanent_uuid: "71eda82db9ac4c4d9fbebfc7d02464ae" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 39667 } }
I20260810 07:18:41.671934 3533 leader_election.cc:288] T ab9e707ab3f344dab9cbee104721a86a P 63b4199da068459a8b85e793b7cf79d2 [CANDIDATE]: Term 1 election: Requested vote from peers f112dbec3d7a4fa5a3754c7ee3472281 (127.15.217.67:45561), 71eda82db9ac4c4d9fbebfc7d02464ae (127.15.217.66:39667)
I20260810 07:18:41.672117 3458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab9e707ab3f344dab9cbee104721a86a" candidate_uuid: "63b4199da068459a8b85e793b7cf79d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f112dbec3d7a4fa5a3754c7ee3472281"
I20260810 07:18:41.672147 3312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab9e707ab3f344dab9cbee104721a86a" candidate_uuid: "63b4199da068459a8b85e793b7cf79d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71eda82db9ac4c4d9fbebfc7d02464ae"
I20260810 07:18:41.672209 3458 raft_consensus.cc:3037] T ab9e707ab3f344dab9cbee104721a86a P f112dbec3d7a4fa5a3754c7ee3472281 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:41.672250 3312 raft_consensus.cc:3037] T ab9e707ab3f344dab9cbee104721a86a P 71eda82db9ac4c4d9fbebfc7d02464ae [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:41.673063 3458 raft_consensus.cc:2454] T ab9e707ab3f344dab9cbee104721a86a P f112dbec3d7a4fa5a3754c7ee3472281 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63b4199da068459a8b85e793b7cf79d2 in term 1.
I20260810 07:18:41.673131 3312 raft_consensus.cc:2454] T ab9e707ab3f344dab9cbee104721a86a P 71eda82db9ac4c4d9fbebfc7d02464ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63b4199da068459a8b85e793b7cf79d2 in term 1.
I20260810 07:18:41.673275 3095 leader_election.cc:302] T ab9e707ab3f344dab9cbee104721a86a P 63b4199da068459a8b85e793b7cf79d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63b4199da068459a8b85e793b7cf79d2, f112dbec3d7a4fa5a3754c7ee3472281; no voters:
I20260810 07:18:41.673391 3533 raft_consensus.cc:2781] T ab9e707ab3f344dab9cbee104721a86a P 63b4199da068459a8b85e793b7cf79d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:18:41.673583 3533 raft_consensus.cc:686] T ab9e707ab3f344dab9cbee104721a86a P 63b4199da068459a8b85e793b7cf79d2 [term 1 LEADER]: Becoming Leader. State: Replica: 63b4199da068459a8b85e793b7cf79d2, State: Running, Role: LEADER
I20260810 07:18:41.673720 3533 consensus_queue.cc:237] T ab9e707ab3f344dab9cbee104721a86a P 63b4199da068459a8b85e793b7cf79d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63b4199da068459a8b85e793b7cf79d2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 40615 } } peers { permanent_uuid: "f112dbec3d7a4fa5a3754c7ee3472281" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 45561 } } peers { permanent_uuid: "71eda82db9ac4c4d9fbebfc7d02464ae" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 39667 } }
I20260810 07:18:41.674736 3006 catalog_manager.cc:5495] T ab9e707ab3f344dab9cbee104721a86a P 63b4199da068459a8b85e793b7cf79d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 63b4199da068459a8b85e793b7cf79d2 (127.15.217.65). New cstate: current_term: 1 leader_uuid: "63b4199da068459a8b85e793b7cf79d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63b4199da068459a8b85e793b7cf79d2" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 40615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f112dbec3d7a4fa5a3754c7ee3472281" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 45561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71eda82db9ac4c4d9fbebfc7d02464ae" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 39667 } health_report { overall_health: UNKNOWN } } }
I20260810 07:18:41.681663 3525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:41.679287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56488 (local address 127.15.217.65:40615)
0810 07:18:41.679349 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:41.679351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:41.679369 (+ 18us) server_negotiation.cc:407] Connection header received
0810 07:18:41.679404 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:41.679405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:41.679423 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:41.679497 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:18:41.679604 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.680082 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.680871 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:41.681058 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:41.681114 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:41.681295 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:18:41.681425 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:18:41.681493 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:41.681511 (+ 18us) server_negotiation.cc:299] Negotiation successful
0810 07:18:41.681532 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 07:18:41.683683 3458 raft_consensus.cc:1261] T ab9e707ab3f344dab9cbee104721a86a P f112dbec3d7a4fa5a3754c7ee3472281 [term 1 FOLLOWER]: Refusing update from remote peer 63b4199da068459a8b85e793b7cf79d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:18:41.683686 3312 raft_consensus.cc:1261] T ab9e707ab3f344dab9cbee104721a86a P 71eda82db9ac4c4d9fbebfc7d02464ae [term 1 FOLLOWER]: Refusing update from remote peer 63b4199da068459a8b85e793b7cf79d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:18:41.684051 3533 consensus_queue.cc:1035] T ab9e707ab3f344dab9cbee104721a86a P 63b4199da068459a8b85e793b7cf79d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71eda82db9ac4c4d9fbebfc7d02464ae" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 39667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:18:41.684211 3537 consensus_queue.cc:1035] T ab9e707ab3f344dab9cbee104721a86a P 63b4199da068459a8b85e793b7cf79d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f112dbec3d7a4fa5a3754c7ee3472281" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 45561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:18:41.687453 3542 mvcc.cc:205] Tried to move back new op lower bound from 7316874533613088768 to 7316874533576376320. Current Snapshot: MvccSnapshot[applied={T|T < 7316874533613088768}]
I20260810 07:18:41.689309 3007 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:18:41.689419 3007 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:18:41.690234 3007 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 36135fe3c5ef45728c8fd7dbf6d3e279: Sending DeleteTablet for 3 replicas of tablet ab9e707ab3f344dab9cbee104721a86a
I20260810 07:18:41.690580 3144 tablet_service.cc:1514] Processing DeleteTablet for tablet ab9e707ab3f344dab9cbee104721a86a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:18:41 UTC) from {username='slave'} at 127.0.0.1:56472
I20260810 07:18:41.690644 3292 tablet_service.cc:1514] Processing DeleteTablet for tablet ab9e707ab3f344dab9cbee104721a86a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:18:41 UTC) from {username='slave'} at 127.0.0.1:50934
I20260810 07:18:41.690626 3437 tablet_service.cc:1514] Processing DeleteTablet for tablet ab9e707ab3f344dab9cbee104721a86a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:18:41 UTC) from {username='slave'} at 127.0.0.1:51584
I20260810 07:18:41.690900 3554 tablet_replica.cc:330] T ab9e707ab3f344dab9cbee104721a86a P 71eda82db9ac4c4d9fbebfc7d02464ae: stopping tablet replica
I20260810 07:18:41.690899 3553 tablet_replica.cc:330] T ab9e707ab3f344dab9cbee104721a86a P 63b4199da068459a8b85e793b7cf79d2: stopping tablet replica
I20260810 07:18:41.691011 3553 raft_consensus.cc:2229] T ab9e707ab3f344dab9cbee104721a86a P 63b4199da068459a8b85e793b7cf79d2 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:18:41.691104 3555 tablet_replica.cc:330] T ab9e707ab3f344dab9cbee104721a86a P f112dbec3d7a4fa5a3754c7ee3472281: stopping tablet replica
I20260810 07:18:41.691125 3553 raft_consensus.cc:2258] T ab9e707ab3f344dab9cbee104721a86a P 63b4199da068459a8b85e793b7cf79d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:18:41.691207 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 3076
I20260810 07:18:41.697906 3554 raft_consensus.cc:2229] T ab9e707ab3f344dab9cbee104721a86a P 71eda82db9ac4c4d9fbebfc7d02464ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:18:41.698109 3554 raft_consensus.cc:2258] T ab9e707ab3f344dab9cbee104721a86a P 71eda82db9ac4c4d9fbebfc7d02464ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:18:41.698251 3555 raft_consensus.cc:2229] T ab9e707ab3f344dab9cbee104721a86a P f112dbec3d7a4fa5a3754c7ee3472281 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:18:41.698410 3555 raft_consensus.cc:2258] T ab9e707ab3f344dab9cbee104721a86a P f112dbec3d7a4fa5a3754c7ee3472281 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:18:41.698945 3555 ts_tablet_manager.cc:1905] T ab9e707ab3f344dab9cbee104721a86a P f112dbec3d7a4fa5a3754c7ee3472281: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:18:41.698976 3554 ts_tablet_manager.cc:1905] T ab9e707ab3f344dab9cbee104721a86a P 71eda82db9ac4c4d9fbebfc7d02464ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:18:41.700652 3555 ts_tablet_manager.cc:1918] T ab9e707ab3f344dab9cbee104721a86a P f112dbec3d7a4fa5a3754c7ee3472281: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:18:41.700793 3555 log.cc:1196] T ab9e707ab3f344dab9cbee104721a86a P f112dbec3d7a4fa5a3754c7ee3472281: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-2/wal/wals/ab9e707ab3f344dab9cbee104721a86a
I20260810 07:18:41.701071 3555 ts_tablet_manager.cc:1939] T ab9e707ab3f344dab9cbee104721a86a P f112dbec3d7a4fa5a3754c7ee3472281: Deleting consensus metadata
I20260810 07:18:41.701164 3554 ts_tablet_manager.cc:1918] T ab9e707ab3f344dab9cbee104721a86a P 71eda82db9ac4c4d9fbebfc7d02464ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:18:41.701258 3554 log.cc:1196] T ab9e707ab3f344dab9cbee104721a86a P 71eda82db9ac4c4d9fbebfc7d02464ae: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346081759552-16229-0/minicluster-data/ts-1/wal/wals/ab9e707ab3f344dab9cbee104721a86a
I20260810 07:18:41.701535 3554 ts_tablet_manager.cc:1939] T ab9e707ab3f344dab9cbee104721a86a P 71eda82db9ac4c4d9fbebfc7d02464ae: Deleting consensus metadata
I20260810 07:18:41.701680 2993 catalog_manager.cc:4841] TS f112dbec3d7a4fa5a3754c7ee3472281 (127.15.217.67:45561): tablet ab9e707ab3f344dab9cbee104721a86a (table test-table [id=05e0c027eb1a44ceb5f01095db572a7e]) successfully deleted
I20260810 07:18:41.701990 2992 catalog_manager.cc:4841] TS 71eda82db9ac4c4d9fbebfc7d02464ae (127.15.217.66:39667): tablet ab9e707ab3f344dab9cbee104721a86a (table test-table [id=05e0c027eb1a44ceb5f01095db572a7e]) successfully deleted
W20260810 07:18:41.703651 2991 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.217.65:40615 (error 108)
I20260810 07:18:41.704188 3520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:41.704033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:40615 (local address 127.0.0.1:56494)
0810 07:18:41.704097 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:41.704145 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.65:40615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 07:18:41.704356 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 3223
W20260810 07:18:41.704365 2991 catalog_manager.cc:4584] TS 63b4199da068459a8b85e793b7cf79d2 (127.15.217.65:40615): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ab9e707ab3f344dab9cbee104721a86a: Network error: Client connection negotiation failed: client connection to 127.15.217.65:40615: connect: Connection refused (error 111)
I20260810 07:18:41.712731 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 3372
I20260810 07:18:41.720845 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 2973
2026-08-10T07:18:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (732 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:18:41 dist-test-slave-10pc krb5kdc[3559](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:18:41 dist-test-slave-10pc krb5kdc[3559](info): set up 2 sockets
Aug 10 07:18:41 dist-test-slave-10pc krb5kdc[3559](info): commencing operation
krb5kdc: starting...
W20260810 07:18:42.815157 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:18:42 dist-test-slave-10pc krb5kdc[3559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:18:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:18:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
I20260810 07:18:42.949275 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:39409
--webserver_interface=127.15.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:40223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:39409
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:18:43.042586 3574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:43.042878 3574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:43.042923 3574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:43.044153 3574 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:18:43.044234 3574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:43.044258 3574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:43.044281 3574 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:18:43.044312 3574 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:18:43.046012 3574 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:40223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:39409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:39409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:43.046523 3574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:43.046838 3574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:43.051944 3582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:43.051978 3580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:43.052241 3574 server_base.cc:1034] running on GCE node
W20260810 07:18:43.051963 3579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:43.052661 3574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:43.052959 3574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:43.054131 3574 hybrid_clock.cc:648] HybridClock initialized: now 1786346323054115 us; error 31 us; skew 500 ppm
Aug 10 07:18:43 dist-test-slave-10pc krb5kdc[3559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346323, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:43.057117 3574 init.cc:377] Logged in from keytab as kudu/127.15.217.126@KRBTEST.COM (short username kudu)
I20260810 07:18:43.057633 3574 webserver.cc:460] Webserver started at http://127.15.217.126:45573/ using document root <none> and password file <none>
I20260810 07:18:43.057880 3574 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:43.057951 3574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:43.058120 3574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:43.059283 3574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "e6162193ca1e4e31b6f7ef4e459db3f4"
format_stamp: "Formatted at 2026-08-10 07:18:43 on dist-test-slave-10pc"
server_key: "0883b1ea21dd275c13512b39ec021680"
server_key_iv: "2f165820d4ed94bfeac8ea210a5d41d1"
server_key_version: "encryptionkey@0"
I20260810 07:18:43.059691 3574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "e6162193ca1e4e31b6f7ef4e459db3f4"
format_stamp: "Formatted at 2026-08-10 07:18:43 on dist-test-slave-10pc"
server_key: "0883b1ea21dd275c13512b39ec021680"
server_key_iv: "2f165820d4ed94bfeac8ea210a5d41d1"
server_key_version: "encryptionkey@0"
I20260810 07:18:43.061357 3574 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:18:43.062261 3590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:43.062474 3574 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:18:43.062603 3574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "e6162193ca1e4e31b6f7ef4e459db3f4"
format_stamp: "Formatted at 2026-08-10 07:18:43 on dist-test-slave-10pc"
server_key: "0883b1ea21dd275c13512b39ec021680"
server_key_iv: "2f165820d4ed94bfeac8ea210a5d41d1"
server_key_version: "encryptionkey@0"
I20260810 07:18:43.062700 3574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:43.072695 3574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:43.074817 3574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:43.075002 3574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:43.079383 3574 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:39409
I20260810 07:18:43.079452 3650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:39409 every 8 connection(s)
I20260810 07:18:43.079838 3574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:18:43.080909 3651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:43.083411 3651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6162193ca1e4e31b6f7ef4e459db3f4: Bootstrap starting.
I20260810 07:18:43.084005 3651 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6162193ca1e4e31b6f7ef4e459db3f4: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:43.084259 3651 log.cc:824] T 00000000000000000000000000000000 P e6162193ca1e4e31b6f7ef4e459db3f4: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:43.085068 3651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6162193ca1e4e31b6f7ef4e459db3f4: No bootstrap required, opened a new log
I20260810 07:18:43.086128 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 3574
I20260810 07:18:43.086295 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:18:43.086529 16229 external_mini_cluster.cc:1428] Setting key 22a99bc00bf70d76397b0113c6283caa
I20260810 07:18:43.087288 3651 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6162193ca1e4e31b6f7ef4e459db3f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6162193ca1e4e31b6f7ef4e459db3f4" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 39409 } }
I20260810 07:18:43.087496 3651 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6162193ca1e4e31b6f7ef4e459db3f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:43.087548 3651 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6162193ca1e4e31b6f7ef4e459db3f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6162193ca1e4e31b6f7ef4e459db3f4, State: Initialized, Role: FOLLOWER
I20260810 07:18:43.087700 3651 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6162193ca1e4e31b6f7ef4e459db3f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6162193ca1e4e31b6f7ef4e459db3f4" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 39409 } }
I20260810 07:18:43.087803 3651 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6162193ca1e4e31b6f7ef4e459db3f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:18:43.087844 3651 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6162193ca1e4e31b6f7ef4e459db3f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:18:43.087905 3651 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6162193ca1e4e31b6f7ef4e459db3f4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:43.089162 3651 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6162193ca1e4e31b6f7ef4e459db3f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6162193ca1e4e31b6f7ef4e459db3f4" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 39409 } }
I20260810 07:18:43.089311 3651 leader_election.cc:302] T 00000000000000000000000000000000 P e6162193ca1e4e31b6f7ef4e459db3f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6162193ca1e4e31b6f7ef4e459db3f4; no voters:
I20260810 07:18:43.089538 3651 leader_election.cc:288] T 00000000000000000000000000000000 P e6162193ca1e4e31b6f7ef4e459db3f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:18:43.089566 3659 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6162193ca1e4e31b6f7ef4e459db3f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:18:43.089690 3659 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6162193ca1e4e31b6f7ef4e459db3f4 [term 1 LEADER]: Becoming Leader. State: Replica: e6162193ca1e4e31b6f7ef4e459db3f4, State: Running, Role: LEADER
I20260810 07:18:43.089879 3651 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6162193ca1e4e31b6f7ef4e459db3f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:18:43.089857 3659 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6162193ca1e4e31b6f7ef4e459db3f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6162193ca1e4e31b6f7ef4e459db3f4" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 39409 } }
I20260810 07:18:43.090297 3659 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6162193ca1e4e31b6f7ef4e459db3f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6162193ca1e4e31b6f7ef4e459db3f4. Latest consensus state: current_term: 1 leader_uuid: "e6162193ca1e4e31b6f7ef4e459db3f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6162193ca1e4e31b6f7ef4e459db3f4" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 39409 } } }
I20260810 07:18:43.090472 3659 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6162193ca1e4e31b6f7ef4e459db3f4 [sys.catalog]: This master's current role is: LEADER
I20260810 07:18:43.090775 3670 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:18:43.091078 3660 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6162193ca1e4e31b6f7ef4e459db3f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6162193ca1e4e31b6f7ef4e459db3f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6162193ca1e4e31b6f7ef4e459db3f4" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 39409 } } }
I20260810 07:18:43.091197 3660 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6162193ca1e4e31b6f7ef4e459db3f4 [sys.catalog]: This master's current role is: LEADER
I20260810 07:18:43.091639 3670 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:18:43.093482 3670 catalog_manager.cc:1347] Generated new cluster ID: 510cb38287d143f282128cddc519cc47
I20260810 07:18:43.093575 3670 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 07:18:43 dist-test-slave-10pc krb5kdc[3559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:18:43.098076 3658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:43.087163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54532 (local address 127.15.217.126:39409)
0810 07:18:43.087366 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:43.087375 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:43.087403 (+ 28us) server_negotiation.cc:407] Connection header received
0810 07:18:43.087513 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:43.087539 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:43.087601 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:43.087745 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:43.088655 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.089809 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.090874 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.091575 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.091685 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:43.094524 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:43.094528 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:43.094528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:43.094541 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:43.096324 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:43.096835 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:43.096845 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:43.096845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:43.096956 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:43.097211 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:43.097212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:43.097212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:43.097472 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:43.097673 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:43.097755 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 07:18:43.097904 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":112,"threads_started":1}
I20260810 07:18:43.120075 3670 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:18:43.120766 3670 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:18:43.124477 3670 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6162193ca1e4e31b6f7ef4e459db3f4: Generated new TSK 0
I20260810 07:18:43.124724 3670 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab.
Entry for principal kudu/127.15.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab.
I20260810 07:18:43.164258 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.65:0
--local_ip_for_outbound_sockets=127.15.217.65
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:39409
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:40223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:18:43.279848 3683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:43.280082 3683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:43.280109 3683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:43.281419 3683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:43.281482 3683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:43.281512 3683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.65
I20260810 07:18:43.283104 3683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:40223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:39409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.65
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:43.283329 3683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:43.283540 3683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:18:43.289361 3683 server_base.cc:1034] running on GCE node
W20260810 07:18:43.289515 3691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:43.289372 3689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:43.289414 3688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:43.289940 3683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:43.290257 3683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:43.291435 3683 hybrid_clock.cc:648] HybridClock initialized: now 1786346323291433 us; error 39 us; skew 500 ppm
Aug 10 07:18:43 dist-test-slave-10pc krb5kdc[3559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346323, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:43.294219 3683 init.cc:377] Logged in from keytab as kudu/127.15.217.65@KRBTEST.COM (short username kudu)
I20260810 07:18:43.294718 3683 webserver.cc:460] Webserver started at http://127.15.217.65:45243/ using document root <none> and password file <none>
I20260810 07:18:43.294963 3683 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:43.295042 3683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:43.295186 3683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:43.296355 3683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-0/data/instance:
uuid: "1ba65383be5942b4b2cc103006139eac"
format_stamp: "Formatted at 2026-08-10 07:18:43 on dist-test-slave-10pc"
server_key: "7efa2c8ae14a17898a2e29144de07247"
server_key_iv: "e772ea51b25a134c5a5ea407fe57dd46"
server_key_version: "encryptionkey@0"
I20260810 07:18:43.296839 3683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance:
uuid: "1ba65383be5942b4b2cc103006139eac"
format_stamp: "Formatted at 2026-08-10 07:18:43 on dist-test-slave-10pc"
server_key: "7efa2c8ae14a17898a2e29144de07247"
server_key_iv: "e772ea51b25a134c5a5ea407fe57dd46"
server_key_version: "encryptionkey@0"
I20260810 07:18:43.298311 3683 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:18:43.299165 3699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:43.299402 3683 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:18:43.299521 3683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-0/wal
uuid: "1ba65383be5942b4b2cc103006139eac"
format_stamp: "Formatted at 2026-08-10 07:18:43 on dist-test-slave-10pc"
server_key: "7efa2c8ae14a17898a2e29144de07247"
server_key_iv: "e772ea51b25a134c5a5ea407fe57dd46"
server_key_version: "encryptionkey@0"
I20260810 07:18:43.299603 3683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:43.325949 3683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:43.328104 3683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:43.328305 3683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:43.328608 3683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:18:43.328981 3683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:18:43.329041 3683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:43.329084 3683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:18:43.329123 3683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:43.335625 3683 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.65:41837
I20260810 07:18:43.335686 3827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.65:41837 every 8 connection(s)
I20260810 07:18:43.336030 3683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
Aug 10 07:18:43 dist-test-slave-10pc krb5kdc[3559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346323, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:18:43.342865 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 3683
I20260810 07:18:43.342964 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance
I20260810 07:18:43.343184 16229 external_mini_cluster.cc:1428] Setting key 54d006a0cb603da3a004033e67ca586d
I20260810 07:18:43.346721 3658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:43.336849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:49271 (local address 127.15.217.126:39409)
0810 07:18:43.336922 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:43.336925 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:43.337187 (+ 262us) server_negotiation.cc:407] Connection header received
0810 07:18:43.337710 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:43.337711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:43.337726 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:43.337783 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:43.338824 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.339342 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.340408 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.340623 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.340678 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:43.343113 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:43.343115 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:43.343115 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:43.343124 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:43.345021 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:43.345518 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:43.345520 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:43.345520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:43.345591 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:43.345956 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:43.345958 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:43.345959 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:43.346117 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:43.346188 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:43.346570 (+ 382us) server_negotiation.cc:299] Negotiation successful
0810 07:18:43.346649 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 07:18:43.347078 3830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:43.336830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:39409 (local address 127.15.217.65:49271)
0810 07:18:43.337051 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:43.337092 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:43.337614 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:43.337864 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:43.337869 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:43.338499 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:18:43.338717 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:43.338721 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.339439 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.339440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:43.340322 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:43.340325 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.340700 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.340701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:43.340814 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:43.341367 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:18:43.341377 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:18:43.342982 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:18:43.345143 (+ 2161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:43.345154 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:43.345157 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:43.345421 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:43.345674 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:43.345675 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:43.345676 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:43.345865 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:43.346197 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:18:43.346206 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:18:43.346404 (+ 198us) client_negotiation.cc:769] Sending connection context
0810 07:18:43.346536 (+ 132us) client_negotiation.cc:240] Negotiation successful
0810 07:18:43.346745 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":116,"threads_started":1}
I20260810 07:18:43.347765 3828 heartbeater.cc:347] Connected to a master server at 127.15.217.126:39409
I20260810 07:18:43.347867 3828 heartbeater.cc:464] Registering TS with master...
I20260810 07:18:43.348088 3828 heartbeater.cc:511] Master 127.15.217.126:39409 requested a full tablet report, sending...
I20260810 07:18:43.348748 3607 ts_manager.cc:194] Registered new tserver with Master: 1ba65383be5942b4b2cc103006139eac (127.15.217.65:41837)
I20260810 07:18:43.349759 3607 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.65@KRBTEST.COM'} at 127.15.217.65:49271
WARNING: no policy specified for kudu/127.15.217.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab.
Entry for principal kudu/127.15.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab.
I20260810 07:18:43.375908 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.66:0
--local_ip_for_outbound_sockets=127.15.217.66
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:39409
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:40223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:18:43.464600 3834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:43.464833 3834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:43.464861 3834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:43.466391 3834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:43.466454 3834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:43.466536 3834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.66
I20260810 07:18:43.468333 3834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:40223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:39409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.66
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:43.468657 3834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:43.468885 3834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:43.474308 3840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:43.474303 3843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:43.474303 3841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:43.474416 3834 server_base.cc:1034] running on GCE node
I20260810 07:18:43.474846 3834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:43.475139 3834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:43.476325 3834 hybrid_clock.cc:648] HybridClock initialized: now 1786346323476304 us; error 42 us; skew 500 ppm
Aug 10 07:18:43 dist-test-slave-10pc krb5kdc[3559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346323, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:43.479239 3834 init.cc:377] Logged in from keytab as kudu/127.15.217.66@KRBTEST.COM (short username kudu)
I20260810 07:18:43.479775 3834 webserver.cc:460] Webserver started at http://127.15.217.66:33761/ using document root <none> and password file <none>
I20260810 07:18:43.480058 3834 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:43.480139 3834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:43.480312 3834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:43.481531 3834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-1/data/instance:
uuid: "630f79a2f9e04c9da99687ca908539da"
format_stamp: "Formatted at 2026-08-10 07:18:43 on dist-test-slave-10pc"
server_key: "664cc0ee6118528015b6ca6047237ea4"
server_key_iv: "0535ab53cf956e0916956b0067fa4acd"
server_key_version: "encryptionkey@0"
I20260810 07:18:43.481930 3834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance:
uuid: "630f79a2f9e04c9da99687ca908539da"
format_stamp: "Formatted at 2026-08-10 07:18:43 on dist-test-slave-10pc"
server_key: "664cc0ee6118528015b6ca6047237ea4"
server_key_iv: "0535ab53cf956e0916956b0067fa4acd"
server_key_version: "encryptionkey@0"
I20260810 07:18:43.483358 3834 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:18:43.484207 3851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:43.484436 3834 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:18:43.484599 3834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-1/wal
uuid: "630f79a2f9e04c9da99687ca908539da"
format_stamp: "Formatted at 2026-08-10 07:18:43 on dist-test-slave-10pc"
server_key: "664cc0ee6118528015b6ca6047237ea4"
server_key_iv: "0535ab53cf956e0916956b0067fa4acd"
server_key_version: "encryptionkey@0"
I20260810 07:18:43.484690 3834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:43.494104 3834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:43.496325 3834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:43.496536 3834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:43.496796 3834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:18:43.497156 3834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:18:43.497219 3834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:43.497262 3834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:18:43.497292 3834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:43.503597 3834 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.66:32943
I20260810 07:18:43.503647 3976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.66:32943 every 8 connection(s)
I20260810 07:18:43.504093 3834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
Aug 10 07:18:43 dist-test-slave-10pc krb5kdc[3559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346323, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:18:43.512912 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 3834
I20260810 07:18:43.513017 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance
I20260810 07:18:43.513253 16229 external_mini_cluster.cc:1428] Setting key 4c66eac44b3278aa3f9ce04a6d09548e
I20260810 07:18:43.514412 3658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:43.504913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:52707 (local address 127.15.217.126:39409)
0810 07:18:43.504975 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:43.504977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:43.505240 (+ 263us) server_negotiation.cc:407] Connection header received
0810 07:18:43.505751 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:43.505753 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:43.505765 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:43.505830 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:43.506859 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.507381 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.508490 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.508688 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.508746 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:43.510945 (+ 2199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:43.510947 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:43.510947 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:43.510954 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:43.512320 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:43.513086 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:43.513088 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:43.513088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:43.513154 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:43.513493 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:43.513494 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:43.513494 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:43.513761 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:43.513835 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:43.514204 (+ 369us) server_negotiation.cc:299] Negotiation successful
0810 07:18:43.514310 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 07:18:43.514724 3979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:43.504859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:39409 (local address 127.15.217.66:52707)
0810 07:18:43.505098 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:43.505143 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:43.505647 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:43.505884 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:43.505887 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:43.506493 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:18:43.506743 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:43.506750 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.507489 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.507490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:43.508367 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:43.508369 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.508770 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.508771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:43.508888 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:43.509422 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:18:43.509431 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:18:43.510842 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:18:43.512411 (+ 1569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:43.512419 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:43.512422 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:43.512703 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:43.513267 (+ 564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:43.513268 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:43.513268 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:43.513402 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:43.513849 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:18:43.513857 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:18:43.514043 (+ 186us) client_negotiation.cc:769] Sending connection context
0810 07:18:43.514167 (+ 124us) client_negotiation.cc:240] Negotiation successful
0810 07:18:43.514388 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":116,"threads_started":1}
I20260810 07:18:43.516598 3977 heartbeater.cc:347] Connected to a master server at 127.15.217.126:39409
I20260810 07:18:43.516739 3977 heartbeater.cc:464] Registering TS with master...
I20260810 07:18:43.516968 3977 heartbeater.cc:511] Master 127.15.217.126:39409 requested a full tablet report, sending...
I20260810 07:18:43.517429 3607 ts_manager.cc:194] Registered new tserver with Master: 630f79a2f9e04c9da99687ca908539da (127.15.217.66:32943)
I20260810 07:18:43.517971 3607 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.66@KRBTEST.COM'} at 127.15.217.66:52707
WARNING: no policy specified for kudu/127.15.217.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab.
Entry for principal kudu/127.15.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab.
I20260810 07:18:43.544394 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.67:0
--local_ip_for_outbound_sockets=127.15.217.67
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:39409
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:40223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:18:43.635110 3983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:43.635350 3983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:43.635380 3983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:43.636652 3983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:43.636700 3983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:43.636732 3983 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.67
I20260810 07:18:43.638271 3983 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:40223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:39409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.67
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:43.638492 3983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:43.638692 3983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:43.644075 3993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:43.644196 3991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:43.644078 3989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:43.644389 3983 server_base.cc:1034] running on GCE node
I20260810 07:18:43.644667 3983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:43.644995 3983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:43.646181 3983 hybrid_clock.cc:648] HybridClock initialized: now 1786346323646153 us; error 39 us; skew 500 ppm
Aug 10 07:18:43 dist-test-slave-10pc krb5kdc[3559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346323, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:43.649168 3983 init.cc:377] Logged in from keytab as kudu/127.15.217.67@KRBTEST.COM (short username kudu)
I20260810 07:18:43.649679 3983 webserver.cc:460] Webserver started at http://127.15.217.67:38217/ using document root <none> and password file <none>
I20260810 07:18:43.649951 3983 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:43.650031 3983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:43.650177 3983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:43.651333 3983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-2/data/instance:
uuid: "77c7ad821b0548eeb66e7c0eb6f83593"
format_stamp: "Formatted at 2026-08-10 07:18:43 on dist-test-slave-10pc"
server_key: "f3263cb99aedb8099d8c5818792920e8"
server_key_iv: "6bd83b77fb4860d72113a66decf79982"
server_key_version: "encryptionkey@0"
I20260810 07:18:43.651748 3983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance:
uuid: "77c7ad821b0548eeb66e7c0eb6f83593"
format_stamp: "Formatted at 2026-08-10 07:18:43 on dist-test-slave-10pc"
server_key: "f3263cb99aedb8099d8c5818792920e8"
server_key_iv: "6bd83b77fb4860d72113a66decf79982"
server_key_version: "encryptionkey@0"
I20260810 07:18:43.653318 3983 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:18:43.654194 4002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:43.654549 3983 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:18:43.654672 3983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-2/wal
uuid: "77c7ad821b0548eeb66e7c0eb6f83593"
format_stamp: "Formatted at 2026-08-10 07:18:43 on dist-test-slave-10pc"
server_key: "f3263cb99aedb8099d8c5818792920e8"
server_key_iv: "6bd83b77fb4860d72113a66decf79982"
server_key_version: "encryptionkey@0"
I20260810 07:18:43.654765 3983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:43.680612 3983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:43.683220 3983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:43.683411 3983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:43.683702 3983 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:18:43.684059 3983 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:18:43.684125 3983 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:43.684168 3983 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:18:43.684199 3983 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:43.690954 3983 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.67:40103
I20260810 07:18:43.691087 4124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.67:40103 every 8 connection(s)
I20260810 07:18:43.691473 3983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
Aug 10 07:18:43 dist-test-slave-10pc krb5kdc[3559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346323, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:18:43.701372 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 3983
I20260810 07:18:43.701507 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance
I20260810 07:18:43.701635 3658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:43.692382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:33171 (local address 127.15.217.126:39409)
0810 07:18:43.692482 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:43.692484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:43.692640 (+ 156us) server_negotiation.cc:407] Connection header received
0810 07:18:43.693143 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:43.693144 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:43.693161 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:43.693253 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:43.694263 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.694831 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.695916 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.696119 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.696217 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:43.698551 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:43.698553 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:43.698553 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:43.698561 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:43.699935 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:43.700337 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:43.700337 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:43.700338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:43.700397 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:43.700714 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:43.700715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:43.700715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:43.700848 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:43.700918 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:43.701381 (+ 463us) server_negotiation.cc:299] Negotiation successful
0810 07:18:43.701498 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 07:18:43.701714 16229 external_mini_cluster.cc:1428] Setting key d90c1693b0c79223b7a6723253030ac2
I20260810 07:18:43.701814 4128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:43.692286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:39409 (local address 127.15.217.67:33171)
0810 07:18:43.692487 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:43.692542 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:43.693053 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:43.693324 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:43.693327 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:43.693939 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:18:43.694154 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:43.694159 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.694926 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.694927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:43.695830 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:43.695832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.696201 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.696202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:43.696307 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:43.696950 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:18:43.696960 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:18:43.698448 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:18:43.700021 (+ 1573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:43.700030 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:43.700033 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:43.700259 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:43.700532 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:43.700533 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:43.700533 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:43.700642 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:43.700960 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:18:43.700968 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:18:43.701204 (+ 236us) client_negotiation.cc:769] Sending connection context
0810 07:18:43.701336 (+ 132us) client_negotiation.cc:240] Negotiation successful
0810 07:18:43.701545 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":99,"threads_started":1}
I20260810 07:18:43.702548 4126 heartbeater.cc:347] Connected to a master server at 127.15.217.126:39409
I20260810 07:18:43.702693 4126 heartbeater.cc:464] Registering TS with master...
I20260810 07:18:43.702934 4126 heartbeater.cc:511] Master 127.15.217.126:39409 requested a full tablet report, sending...
I20260810 07:18:43.703411 3607 ts_manager.cc:194] Registered new tserver with Master: 77c7ad821b0548eeb66e7c0eb6f83593 (127.15.217.67:40103)
I20260810 07:18:43.703850 3607 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.67@KRBTEST.COM'} at 127.15.217.67:33171
I20260810 07:18:43.714030 16229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:18:43.722463 3658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:43.715544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54534 (local address 127.15.217.126:39409)
0810 07:18:43.715619 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:43.715623 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:43.715715 (+ 92us) server_negotiation.cc:407] Connection header received
0810 07:18:43.715779 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:43.715779 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:43.715789 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:43.715839 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:43.716691 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.717203 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.717956 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.718144 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.718195 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:43.719753 (+ 1558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:43.719754 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:43.719755 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:43.719762 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:43.721160 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:43.721599 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:43.721599 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:43.721600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:43.721669 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:43.721930 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:43.721930 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:43.721930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:43.722066 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:43.722148 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:43.722281 (+ 133us) server_negotiation.cc:299] Negotiation successful
0810 07:18:43.722338 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 07:18:43.730116 3607 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54534:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:18:43.731813 3607 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:18:43.739115 4141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:43.734629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33856 (local address 127.15.217.67:40103)
0810 07:18:43.734841 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:43.734847 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:43.735254 (+ 407us) server_negotiation.cc:407] Connection header received
0810 07:18:43.735302 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:43.735304 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:43.735320 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:43.735387 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:43.735587 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.736794 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.738004 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.738760 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.738841 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:43.738901 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:43.738978 (+ 77us) server_negotiation.cc:299] Negotiation successful
0810 07:18:43.739025 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":91,"threads_started":1}
I20260810 07:18:43.739094 4143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:43.734551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:40103 (local address 127.0.0.1:33856)
0810 07:18:43.735114 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:43.735125 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:43.735189 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:43.735463 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:43.735464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:43.735466 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:43.735505 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:43.735506 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.736950 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.736951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:43.737891 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:43.737894 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.738846 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.738847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:43.738907 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:43.738911 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:18:43.738963 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 07:18:43.738992 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":113,"threads_started":1}
I20260810 07:18:43.739115 4138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:43.734779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:32943 (local address 127.0.0.1:55684)
0810 07:18:43.735197 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:43.735207 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:43.735247 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:43.735546 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:43.735547 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:43.735549 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:43.735601 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:43.735603 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.736574 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.736575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:43.737930 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:43.737933 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.738862 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.738863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:43.738931 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:43.738946 (+ 15us) client_negotiation.cc:769] Sending connection context
0810 07:18:43.738983 (+ 37us) client_negotiation.cc:240] Negotiation successful
0810 07:18:43.739027 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"mutex_wait_us":35,"thread_start_us":72,"threads_started":1}
I20260810 07:18:43.739094 4140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:43.734549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:41837 (local address 127.0.0.1:54926)
0810 07:18:43.734939 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:43.734974 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:43.735075 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:43.735319 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:43.735322 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:43.735325 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:43.735426 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:43.735429 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.736915 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.736916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:43.738004 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:43.738006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.738695 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.738696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:43.738788 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:43.738804 (+ 16us) client_negotiation.cc:769] Sending connection context
0810 07:18:43.738878 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 07:18:43.738966 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":70,"threads_started":1}
I20260810 07:18:43.739455 4144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:43.735085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55684 (local address 127.15.217.66:32943)
0810 07:18:43.735373 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:43.735378 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:43.735388 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:18:43.735431 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:43.735433 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:43.735448 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:43.735693 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:43.735769 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.736437 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.738052 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.738760 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.739162 (+ 402us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:43.739244 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:43.739276 (+ 32us) server_negotiation.cc:299] Negotiation successful
0810 07:18:43.739349 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":127,"threads_started":1}
I20260810 07:18:43.739532 4142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:43.734724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54926 (local address 127.15.217.65:41837)
0810 07:18:43.734925 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:43.734930 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:43.735076 (+ 146us) server_negotiation.cc:407] Connection header received
0810 07:18:43.735177 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:43.735180 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:43.735198 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:43.735527 (+ 329us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:43.735607 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.736792 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.738071 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.738588 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.738695 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:43.738763 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:43.739147 (+ 384us) server_negotiation.cc:299] Negotiation successful
0810 07:18:43.739449 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":99,"threads_started":1}
I20260810 07:18:43.740244 3750 tablet_service.cc:1467] Processing CreateTablet for tablet 2c6712d2c3784ab7b667768a753f2088 (DEFAULT_TABLE table=test-table [id=b95bd322150e4ef4bc1188e1178e46bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:18:43.740260 3900 tablet_service.cc:1467] Processing CreateTablet for tablet 2c6712d2c3784ab7b667768a753f2088 (DEFAULT_TABLE table=test-table [id=b95bd322150e4ef4bc1188e1178e46bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:18:43.740257 4051 tablet_service.cc:1467] Processing CreateTablet for tablet 2c6712d2c3784ab7b667768a753f2088 (DEFAULT_TABLE table=test-table [id=b95bd322150e4ef4bc1188e1178e46bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:18:43.740691 3750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c6712d2c3784ab7b667768a753f2088. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:43.740692 3900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c6712d2c3784ab7b667768a753f2088. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:43.740692 4051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c6712d2c3784ab7b667768a753f2088. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:43.743525 4146 tablet_bootstrap.cc:492] T 2c6712d2c3784ab7b667768a753f2088 P 1ba65383be5942b4b2cc103006139eac: Bootstrap starting.
I20260810 07:18:43.744099 4148 tablet_bootstrap.cc:492] T 2c6712d2c3784ab7b667768a753f2088 P 630f79a2f9e04c9da99687ca908539da: Bootstrap starting.
I20260810 07:18:43.744307 4147 tablet_bootstrap.cc:492] T 2c6712d2c3784ab7b667768a753f2088 P 77c7ad821b0548eeb66e7c0eb6f83593: Bootstrap starting.
I20260810 07:18:43.744349 4146 tablet_bootstrap.cc:654] T 2c6712d2c3784ab7b667768a753f2088 P 1ba65383be5942b4b2cc103006139eac: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:43.744916 4147 tablet_bootstrap.cc:654] T 2c6712d2c3784ab7b667768a753f2088 P 77c7ad821b0548eeb66e7c0eb6f83593: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:43.744904 4148 tablet_bootstrap.cc:654] T 2c6712d2c3784ab7b667768a753f2088 P 630f79a2f9e04c9da99687ca908539da: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:43.745059 4146 log.cc:824] T 2c6712d2c3784ab7b667768a753f2088 P 1ba65383be5942b4b2cc103006139eac: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:43.745579 4148 log.cc:824] T 2c6712d2c3784ab7b667768a753f2088 P 630f79a2f9e04c9da99687ca908539da: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:43.745642 4147 log.cc:824] T 2c6712d2c3784ab7b667768a753f2088 P 77c7ad821b0548eeb66e7c0eb6f83593: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:43.746057 4146 tablet_bootstrap.cc:492] T 2c6712d2c3784ab7b667768a753f2088 P 1ba65383be5942b4b2cc103006139eac: No bootstrap required, opened a new log
I20260810 07:18:43.746202 4146 ts_tablet_manager.cc:1397] T 2c6712d2c3784ab7b667768a753f2088 P 1ba65383be5942b4b2cc103006139eac: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:18:43.746480 4148 tablet_bootstrap.cc:492] T 2c6712d2c3784ab7b667768a753f2088 P 630f79a2f9e04c9da99687ca908539da: No bootstrap required, opened a new log
I20260810 07:18:43.746491 4147 tablet_bootstrap.cc:492] T 2c6712d2c3784ab7b667768a753f2088 P 77c7ad821b0548eeb66e7c0eb6f83593: No bootstrap required, opened a new log
I20260810 07:18:43.746562 4147 ts_tablet_manager.cc:1397] T 2c6712d2c3784ab7b667768a753f2088 P 77c7ad821b0548eeb66e7c0eb6f83593: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:18:43.746567 4148 ts_tablet_manager.cc:1397] T 2c6712d2c3784ab7b667768a753f2088 P 630f79a2f9e04c9da99687ca908539da: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:18:43.747762 4146 raft_consensus.cc:348] T 2c6712d2c3784ab7b667768a753f2088 P 1ba65383be5942b4b2cc103006139eac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ba65383be5942b4b2cc103006139eac" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41837 } } peers { permanent_uuid: "77c7ad821b0548eeb66e7c0eb6f83593" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40103 } } peers { permanent_uuid: "630f79a2f9e04c9da99687ca908539da" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 32943 } }
I20260810 07:18:43.747902 4146 raft_consensus.cc:374] T 2c6712d2c3784ab7b667768a753f2088 P 1ba65383be5942b4b2cc103006139eac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:43.747923 4146 raft_consensus.cc:729] T 2c6712d2c3784ab7b667768a753f2088 P 1ba65383be5942b4b2cc103006139eac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ba65383be5942b4b2cc103006139eac, State: Initialized, Role: FOLLOWER
I20260810 07:18:43.748059 4146 consensus_queue.cc:260] T 2c6712d2c3784ab7b667768a753f2088 P 1ba65383be5942b4b2cc103006139eac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ba65383be5942b4b2cc103006139eac" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41837 } } peers { permanent_uuid: "77c7ad821b0548eeb66e7c0eb6f83593" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40103 } } peers { permanent_uuid: "630f79a2f9e04c9da99687ca908539da" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 32943 } }
I20260810 07:18:43.748404 3828 heartbeater.cc:503] Master 127.15.217.126:39409 was elected leader, sending a full tablet report...
I20260810 07:18:43.748616 4147 raft_consensus.cc:348] T 2c6712d2c3784ab7b667768a753f2088 P 77c7ad821b0548eeb66e7c0eb6f83593 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ba65383be5942b4b2cc103006139eac" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41837 } } peers { permanent_uuid: "77c7ad821b0548eeb66e7c0eb6f83593" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40103 } } peers { permanent_uuid: "630f79a2f9e04c9da99687ca908539da" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 32943 } }
I20260810 07:18:43.748622 4148 raft_consensus.cc:348] T 2c6712d2c3784ab7b667768a753f2088 P 630f79a2f9e04c9da99687ca908539da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ba65383be5942b4b2cc103006139eac" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41837 } } peers { permanent_uuid: "77c7ad821b0548eeb66e7c0eb6f83593" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40103 } } peers { permanent_uuid: "630f79a2f9e04c9da99687ca908539da" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 32943 } }
I20260810 07:18:43.748780 4146 ts_tablet_manager.cc:1428] T 2c6712d2c3784ab7b667768a753f2088 P 1ba65383be5942b4b2cc103006139eac: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:18:43.748797 4147 raft_consensus.cc:374] T 2c6712d2c3784ab7b667768a753f2088 P 77c7ad821b0548eeb66e7c0eb6f83593 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:43.748797 4148 raft_consensus.cc:374] T 2c6712d2c3784ab7b667768a753f2088 P 630f79a2f9e04c9da99687ca908539da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:43.748837 4148 raft_consensus.cc:729] T 2c6712d2c3784ab7b667768a753f2088 P 630f79a2f9e04c9da99687ca908539da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 630f79a2f9e04c9da99687ca908539da, State: Initialized, Role: FOLLOWER
I20260810 07:18:43.748837 4147 raft_consensus.cc:729] T 2c6712d2c3784ab7b667768a753f2088 P 77c7ad821b0548eeb66e7c0eb6f83593 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77c7ad821b0548eeb66e7c0eb6f83593, State: Initialized, Role: FOLLOWER
I20260810 07:18:43.748966 4147 consensus_queue.cc:260] T 2c6712d2c3784ab7b667768a753f2088 P 77c7ad821b0548eeb66e7c0eb6f83593 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ba65383be5942b4b2cc103006139eac" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41837 } } peers { permanent_uuid: "77c7ad821b0548eeb66e7c0eb6f83593" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40103 } } peers { permanent_uuid: "630f79a2f9e04c9da99687ca908539da" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 32943 } }
I20260810 07:18:43.748956 4148 consensus_queue.cc:260] T 2c6712d2c3784ab7b667768a753f2088 P 630f79a2f9e04c9da99687ca908539da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ba65383be5942b4b2cc103006139eac" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41837 } } peers { permanent_uuid: "77c7ad821b0548eeb66e7c0eb6f83593" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40103 } } peers { permanent_uuid: "630f79a2f9e04c9da99687ca908539da" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 32943 } }
I20260810 07:18:43.749261 4147 ts_tablet_manager.cc:1428] T 2c6712d2c3784ab7b667768a753f2088 P 77c7ad821b0548eeb66e7c0eb6f83593: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:18:43.749293 4126 heartbeater.cc:503] Master 127.15.217.126:39409 was elected leader, sending a full tablet report...
I20260810 07:18:43.749507 4148 ts_tablet_manager.cc:1428] T 2c6712d2c3784ab7b667768a753f2088 P 630f79a2f9e04c9da99687ca908539da: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:18:43.749642 3977 heartbeater.cc:503] Master 127.15.217.126:39409 was elected leader, sending a full tablet report...
W20260810 07:18:43.754642 3978 tablet.cc:2369] T 2c6712d2c3784ab7b667768a753f2088 P 630f79a2f9e04c9da99687ca908539da: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:18:43.791733 4154 raft_consensus.cc:482] T 2c6712d2c3784ab7b667768a753f2088 P 77c7ad821b0548eeb66e7c0eb6f83593 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:18:43.791882 4154 raft_consensus.cc:504] T 2c6712d2c3784ab7b667768a753f2088 P 77c7ad821b0548eeb66e7c0eb6f83593 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ba65383be5942b4b2cc103006139eac" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41837 } } peers { permanent_uuid: "77c7ad821b0548eeb66e7c0eb6f83593" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40103 } } peers { permanent_uuid: "630f79a2f9e04c9da99687ca908539da" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 32943 } }
I20260810 07:18:43.792315 4154 leader_election.cc:288] T 2c6712d2c3784ab7b667768a753f2088 P 77c7ad821b0548eeb66e7c0eb6f83593 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ba65383be5942b4b2cc103006139eac (127.15.217.65:41837), 630f79a2f9e04c9da99687ca908539da (127.15.217.66:32943)
I20260810 07:18:43.795550 4142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:43.792666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:39223 (local address 127.15.217.65:41837)
0810 07:18:43.792714 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:43.792716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:43.792727 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:18:43.792799 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:43.792800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:43.792813 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:43.792869 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:43.793102 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.793654 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.794699 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.795127 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.795189 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:43.795197 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:43.795407 (+ 210us) server_negotiation.cc:299] Negotiation successful
0810 07:18:43.795460 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 07:18:43.795588 4144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:43.792820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:54835 (local address 127.15.217.66:32943)
0810 07:18:43.792874 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:43.792877 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:43.792929 (+ 52us) server_negotiation.cc:407] Connection header received
0810 07:18:43.792990 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:43.792991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:43.793005 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:43.793080 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:43.793259 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.793781 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.794773 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.795189 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.795267 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:43.795277 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:43.795472 (+ 195us) server_negotiation.cc:299] Negotiation successful
0810 07:18:43.795518 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 07:18:43.795535 4128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:43.792547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:41837 (local address 127.15.217.67:39223)
0810 07:18:43.792633 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:43.792658 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:43.792732 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:43.792911 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:43.792912 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:43.792915 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:43.793007 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:43.793009 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.793757 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.793758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:43.794585 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:43.794587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.795222 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.795223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:43.795318 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:43.795321 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:18:43.795380 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 07:18:43.795439 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 07:18:43.795894 4155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:43.792687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:32943 (local address 127.15.217.67:54835)
0810 07:18:43.792873 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:43.792884 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:43.792934 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:43.793098 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:43.793099 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:43.793101 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:43.793176 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:43.793177 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.793868 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.793869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:43.794667 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:43.794670 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.795287 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.795288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:43.795375 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:43.795380 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:18:43.795441 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 07:18:43.795478 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":128,"threads_started":1}
I20260810 07:18:43.796034 3772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c6712d2c3784ab7b667768a753f2088" candidate_uuid: "77c7ad821b0548eeb66e7c0eb6f83593" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ba65383be5942b4b2cc103006139eac" is_pre_election: true
I20260810 07:18:43.796253 3772 raft_consensus.cc:2454] T 2c6712d2c3784ab7b667768a753f2088 P 1ba65383be5942b4b2cc103006139eac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77c7ad821b0548eeb66e7c0eb6f83593 in term 0.
I20260810 07:18:43.796470 3925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c6712d2c3784ab7b667768a753f2088" candidate_uuid: "77c7ad821b0548eeb66e7c0eb6f83593" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "630f79a2f9e04c9da99687ca908539da" is_pre_election: true
I20260810 07:18:43.796554 4005 leader_election.cc:302] T 2c6712d2c3784ab7b667768a753f2088 P 77c7ad821b0548eeb66e7c0eb6f83593 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ba65383be5942b4b2cc103006139eac, 77c7ad821b0548eeb66e7c0eb6f83593; no voters:
I20260810 07:18:43.796633 3925 raft_consensus.cc:2454] T 2c6712d2c3784ab7b667768a753f2088 P 630f79a2f9e04c9da99687ca908539da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77c7ad821b0548eeb66e7c0eb6f83593 in term 0.
I20260810 07:18:43.796723 4154 raft_consensus.cc:2781] T 2c6712d2c3784ab7b667768a753f2088 P 77c7ad821b0548eeb66e7c0eb6f83593 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:18:43.796782 4154 raft_consensus.cc:482] T 2c6712d2c3784ab7b667768a753f2088 P 77c7ad821b0548eeb66e7c0eb6f83593 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:18:43.796830 4154 raft_consensus.cc:3037] T 2c6712d2c3784ab7b667768a753f2088 P 77c7ad821b0548eeb66e7c0eb6f83593 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:43.797675 4154 raft_consensus.cc:504] T 2c6712d2c3784ab7b667768a753f2088 P 77c7ad821b0548eeb66e7c0eb6f83593 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ba65383be5942b4b2cc103006139eac" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41837 } } peers { permanent_uuid: "77c7ad821b0548eeb66e7c0eb6f83593" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40103 } } peers { permanent_uuid: "630f79a2f9e04c9da99687ca908539da" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 32943 } }
I20260810 07:18:43.797801 4154 leader_election.cc:288] T 2c6712d2c3784ab7b667768a753f2088 P 77c7ad821b0548eeb66e7c0eb6f83593 [CANDIDATE]: Term 1 election: Requested vote from peers 1ba65383be5942b4b2cc103006139eac (127.15.217.65:41837), 630f79a2f9e04c9da99687ca908539da (127.15.217.66:32943)
I20260810 07:18:43.798043 3772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c6712d2c3784ab7b667768a753f2088" candidate_uuid: "77c7ad821b0548eeb66e7c0eb6f83593" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ba65383be5942b4b2cc103006139eac"
I20260810 07:18:43.798094 3925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c6712d2c3784ab7b667768a753f2088" candidate_uuid: "77c7ad821b0548eeb66e7c0eb6f83593" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "630f79a2f9e04c9da99687ca908539da"
I20260810 07:18:43.798169 3925 raft_consensus.cc:3037] T 2c6712d2c3784ab7b667768a753f2088 P 630f79a2f9e04c9da99687ca908539da [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:43.798174 3772 raft_consensus.cc:3037] T 2c6712d2c3784ab7b667768a753f2088 P 1ba65383be5942b4b2cc103006139eac [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:43.798945 3925 raft_consensus.cc:2454] T 2c6712d2c3784ab7b667768a753f2088 P 630f79a2f9e04c9da99687ca908539da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77c7ad821b0548eeb66e7c0eb6f83593 in term 1.
I20260810 07:18:43.799101 3772 raft_consensus.cc:2454] T 2c6712d2c3784ab7b667768a753f2088 P 1ba65383be5942b4b2cc103006139eac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77c7ad821b0548eeb66e7c0eb6f83593 in term 1.
I20260810 07:18:43.799206 4005 leader_election.cc:302] T 2c6712d2c3784ab7b667768a753f2088 P 77c7ad821b0548eeb66e7c0eb6f83593 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 630f79a2f9e04c9da99687ca908539da, 77c7ad821b0548eeb66e7c0eb6f83593; no voters:
I20260810 07:18:43.799301 4154 raft_consensus.cc:2781] T 2c6712d2c3784ab7b667768a753f2088 P 77c7ad821b0548eeb66e7c0eb6f83593 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:18:43.799475 4154 raft_consensus.cc:686] T 2c6712d2c3784ab7b667768a753f2088 P 77c7ad821b0548eeb66e7c0eb6f83593 [term 1 LEADER]: Becoming Leader. State: Replica: 77c7ad821b0548eeb66e7c0eb6f83593, State: Running, Role: LEADER
I20260810 07:18:43.799602 4154 consensus_queue.cc:237] T 2c6712d2c3784ab7b667768a753f2088 P 77c7ad821b0548eeb66e7c0eb6f83593 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ba65383be5942b4b2cc103006139eac" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41837 } } peers { permanent_uuid: "77c7ad821b0548eeb66e7c0eb6f83593" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40103 } } peers { permanent_uuid: "630f79a2f9e04c9da99687ca908539da" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 32943 } }
I20260810 07:18:43.800730 3606 catalog_manager.cc:5495] T 2c6712d2c3784ab7b667768a753f2088 P 77c7ad821b0548eeb66e7c0eb6f83593 reported cstate change: term changed from 0 to 1, leader changed from <none> to 77c7ad821b0548eeb66e7c0eb6f83593 (127.15.217.67). New cstate: current_term: 1 leader_uuid: "77c7ad821b0548eeb66e7c0eb6f83593" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ba65383be5942b4b2cc103006139eac" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "77c7ad821b0548eeb66e7c0eb6f83593" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 40103 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "630f79a2f9e04c9da99687ca908539da" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 32943 } health_report { overall_health: UNKNOWN } } }
I20260810 07:18:43.817737 4141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:43.814909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33860 (local address 127.15.217.67:40103)
0810 07:18:43.814987 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:43.814989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:43.815004 (+ 15us) server_negotiation.cc:407] Connection header received
0810 07:18:43.815060 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:43.815061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:43.815082 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:43.815173 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:18:43.815350 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.815918 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.816838 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.817077 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.817165 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:43.817329 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:18:43.817442 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:18:43.817565 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:43.817597 (+ 32us) server_negotiation.cc:299] Negotiation successful
0810 07:18:43.817610 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 07:18:43.819972 3772 raft_consensus.cc:1261] T 2c6712d2c3784ab7b667768a753f2088 P 1ba65383be5942b4b2cc103006139eac [term 1 FOLLOWER]: Refusing update from remote peer 77c7ad821b0548eeb66e7c0eb6f83593: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:18:43.819972 3925 raft_consensus.cc:1261] T 2c6712d2c3784ab7b667768a753f2088 P 630f79a2f9e04c9da99687ca908539da [term 1 FOLLOWER]: Refusing update from remote peer 77c7ad821b0548eeb66e7c0eb6f83593: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:18:43.820391 4154 consensus_queue.cc:1035] T 2c6712d2c3784ab7b667768a753f2088 P 77c7ad821b0548eeb66e7c0eb6f83593 [LEADER]: Connected to new peer: Peer: permanent_uuid: "630f79a2f9e04c9da99687ca908539da" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 32943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:18:43.820623 4156 consensus_queue.cc:1035] T 2c6712d2c3784ab7b667768a753f2088 P 77c7ad821b0548eeb66e7c0eb6f83593 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ba65383be5942b4b2cc103006139eac" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 41837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:18:43.824514 4159 mvcc.cc:205] Tried to move back new op lower bound from 7316874542363045888 to 7316874542284017664. Current Snapshot: MvccSnapshot[applied={T|T < 7316874542363045888}]
I20260810 07:18:43.827210 3606 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:18:43.827343 3606 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:18:43.828341 3606 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e6162193ca1e4e31b6f7ef4e459db3f4: Sending DeleteTablet for 3 replicas of tablet 2c6712d2c3784ab7b667768a753f2088
I20260810 07:18:43.828786 4051 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c6712d2c3784ab7b667768a753f2088 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:18:43 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:33856
I20260810 07:18:43.828830 3750 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c6712d2c3784ab7b667768a753f2088 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:18:43 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:54926
I20260810 07:18:43.829149 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 3683
I20260810 07:18:43.829180 4172 tablet_replica.cc:330] T 2c6712d2c3784ab7b667768a753f2088 P 1ba65383be5942b4b2cc103006139eac: stopping tablet replica
I20260810 07:18:43.829197 3900 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c6712d2c3784ab7b667768a753f2088 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:18:43 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:55684
I20260810 07:18:43.829289 4172 raft_consensus.cc:2229] T 2c6712d2c3784ab7b667768a753f2088 P 1ba65383be5942b4b2cc103006139eac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:18:43.829404 4172 raft_consensus.cc:2258] T 2c6712d2c3784ab7b667768a753f2088 P 1ba65383be5942b4b2cc103006139eac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:18:43.829577 4171 tablet_replica.cc:330] T 2c6712d2c3784ab7b667768a753f2088 P 77c7ad821b0548eeb66e7c0eb6f83593: stopping tablet replica
I20260810 07:18:43.829669 4171 raft_consensus.cc:2229] T 2c6712d2c3784ab7b667768a753f2088 P 77c7ad821b0548eeb66e7c0eb6f83593 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:18:43.829795 4171 raft_consensus.cc:2258] T 2c6712d2c3784ab7b667768a753f2088 P 77c7ad821b0548eeb66e7c0eb6f83593 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:18:43.831292 4171 ts_tablet_manager.cc:1905] T 2c6712d2c3784ab7b667768a753f2088 P 77c7ad821b0548eeb66e7c0eb6f83593: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:18:43.833639 4171 ts_tablet_manager.cc:1918] T 2c6712d2c3784ab7b667768a753f2088 P 77c7ad821b0548eeb66e7c0eb6f83593: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:18:43.833741 4171 log.cc:1196] T 2c6712d2c3784ab7b667768a753f2088 P 77c7ad821b0548eeb66e7c0eb6f83593: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346081759552-16229-0/minicluster-data/ts-2/wal/wals/2c6712d2c3784ab7b667768a753f2088
I20260810 07:18:43.834036 4171 ts_tablet_manager.cc:1939] T 2c6712d2c3784ab7b667768a753f2088 P 77c7ad821b0548eeb66e7c0eb6f83593: Deleting consensus metadata
I20260810 07:18:43.834594 3594 catalog_manager.cc:4841] TS 77c7ad821b0548eeb66e7c0eb6f83593 (127.15.217.67:40103): tablet 2c6712d2c3784ab7b667768a753f2088 (table test-table [id=b95bd322150e4ef4bc1188e1178e46bd]) successfully deleted
W20260810 07:18:43.839570 3593 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.217.65:41837 (error 108)
I20260810 07:18:43.840240 4173 tablet_replica.cc:330] T 2c6712d2c3784ab7b667768a753f2088 P 630f79a2f9e04c9da99687ca908539da: stopping tablet replica
I20260810 07:18:43.840291 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 3834
I20260810 07:18:43.840404 4173 raft_consensus.cc:2229] T 2c6712d2c3784ab7b667768a753f2088 P 630f79a2f9e04c9da99687ca908539da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:18:43.840565 4173 raft_consensus.cc:2258] T 2c6712d2c3784ab7b667768a753f2088 P 630f79a2f9e04c9da99687ca908539da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:18:43.843477 4143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:43.843205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:41837 (local address 127.0.0.1:54936)
0810 07:18:43.843354 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:43.843422 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.65:41837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
W20260810 07:18:43.843689 3593 catalog_manager.cc:4584] TS 1ba65383be5942b4b2cc103006139eac (127.15.217.65:41837): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2c6712d2c3784ab7b667768a753f2088: Network error: Client connection negotiation failed: client connection to 127.15.217.65:41837: connect: Connection refused (error 111)
I20260810 07:18:43.850044 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 3983
I20260810 07:18:43.850260 4143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:43.850101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:32943 (local address 127.0.0.1:55688)
0810 07:18:43.850173 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:43.850208 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.66:32943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 07:18:43.858891 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 3574
2026-08-10T07:18:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2140 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-10T07:18:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:18:43Z Disabled control of system clock
I20260810 07:18:43.893224 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:44657
--webserver_interface=127.15.217.126
--webserver_port=0
--builtin_ntp_servers=127.15.217.84:41179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:44657
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 07:18:43.983388 4179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:43.983669 4179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:43.983696 4179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:43.985109 4179 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:18:43.985162 4179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:43.985183 4179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:43.985195 4179 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:18:43.985208 4179 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:18:43.986690 4179 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:41179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:44657
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:44657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:43.987002 4179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:43.987363 4179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:43.992822 4185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:43.992808 4184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:43.992808 4187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:43.992861 4179 server_base.cc:1034] running on GCE node
I20260810 07:18:43.993292 4179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:43.993567 4179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:43.994757 4179 hybrid_clock.cc:648] HybridClock initialized: now 1786346323994734 us; error 56 us; skew 500 ppm
I20260810 07:18:43.996122 4179 webserver.cc:460] Webserver started at http://127.15.217.126:40079/ using document root <none> and password file <none>
I20260810 07:18:43.996429 4179 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:43.996568 4179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:43.996770 4179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:43.997939 4179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "2b7916476c6b4d3daa4172f934584877"
format_stamp: "Formatted at 2026-08-10 07:18:43 on dist-test-slave-10pc"
server_key: "3c3d0e2529cad4ef3da35ec0ed734a47"
server_key_iv: "5b5ef32d0277c19c1f27b7d1cd01de3b"
server_key_version: "encryptionkey@0"
I20260810 07:18:43.998371 4179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "2b7916476c6b4d3daa4172f934584877"
format_stamp: "Formatted at 2026-08-10 07:18:43 on dist-test-slave-10pc"
server_key: "3c3d0e2529cad4ef3da35ec0ed734a47"
server_key_iv: "5b5ef32d0277c19c1f27b7d1cd01de3b"
server_key_version: "encryptionkey@0"
I20260810 07:18:43.999804 4179 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:18:44.000857 4193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:44.001077 4179 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:18:44.001186 4179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "2b7916476c6b4d3daa4172f934584877"
format_stamp: "Formatted at 2026-08-10 07:18:43 on dist-test-slave-10pc"
server_key: "3c3d0e2529cad4ef3da35ec0ed734a47"
server_key_iv: "5b5ef32d0277c19c1f27b7d1cd01de3b"
server_key_version: "encryptionkey@0"
I20260810 07:18:44.001291 4179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:44.024757 4179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:44.025175 4179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:44.025362 4179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:44.030720 4179 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:44657
I20260810 07:18:44.030757 4251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:44657 every 8 connection(s)
I20260810 07:18:44.031164 4179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:18:44.032306 4252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:44.034807 4252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b7916476c6b4d3daa4172f934584877: Bootstrap starting.
I20260810 07:18:44.035461 4252 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b7916476c6b4d3daa4172f934584877: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:44.035794 4252 log.cc:824] T 00000000000000000000000000000000 P 2b7916476c6b4d3daa4172f934584877: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:44.036657 4252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b7916476c6b4d3daa4172f934584877: No bootstrap required, opened a new log
I20260810 07:18:44.038357 4252 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b7916476c6b4d3daa4172f934584877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b7916476c6b4d3daa4172f934584877" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 44657 } }
I20260810 07:18:44.038508 4252 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b7916476c6b4d3daa4172f934584877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:44.038532 4252 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b7916476c6b4d3daa4172f934584877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b7916476c6b4d3daa4172f934584877, State: Initialized, Role: FOLLOWER
I20260810 07:18:44.038712 4252 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b7916476c6b4d3daa4172f934584877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b7916476c6b4d3daa4172f934584877" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 44657 } }
I20260810 07:18:44.038785 4252 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b7916476c6b4d3daa4172f934584877 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:18:44.038808 4252 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b7916476c6b4d3daa4172f934584877 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:18:44.038885 4252 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b7916476c6b4d3daa4172f934584877 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:44.039109 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 4179
I20260810 07:18:44.039240 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:18:44.039449 16229 external_mini_cluster.cc:1428] Setting key 1617240f03e0fec5178974eac759606d
I20260810 07:18:44.039563 4252 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b7916476c6b4d3daa4172f934584877 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b7916476c6b4d3daa4172f934584877" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 44657 } }
I20260810 07:18:44.039709 4252 leader_election.cc:302] T 00000000000000000000000000000000 P 2b7916476c6b4d3daa4172f934584877 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b7916476c6b4d3daa4172f934584877; no voters:
I20260810 07:18:44.039955 4252 leader_election.cc:288] T 00000000000000000000000000000000 P 2b7916476c6b4d3daa4172f934584877 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:18:44.040053 4255 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b7916476c6b4d3daa4172f934584877 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:18:44.040251 4255 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b7916476c6b4d3daa4172f934584877 [term 1 LEADER]: Becoming Leader. State: Replica: 2b7916476c6b4d3daa4172f934584877, State: Running, Role: LEADER
I20260810 07:18:44.040324 4252 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b7916476c6b4d3daa4172f934584877 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:18:44.040373 4255 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b7916476c6b4d3daa4172f934584877 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b7916476c6b4d3daa4172f934584877" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 44657 } }
I20260810 07:18:44.040727 4255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b7916476c6b4d3daa4172f934584877 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b7916476c6b4d3daa4172f934584877. Latest consensus state: current_term: 1 leader_uuid: "2b7916476c6b4d3daa4172f934584877" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b7916476c6b4d3daa4172f934584877" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 44657 } } }
I20260810 07:18:44.040846 4259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b7916476c6b4d3daa4172f934584877 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b7916476c6b4d3daa4172f934584877" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b7916476c6b4d3daa4172f934584877" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 44657 } } }
I20260810 07:18:44.041003 4259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b7916476c6b4d3daa4172f934584877 [sys.catalog]: This master's current role is: LEADER
I20260810 07:18:44.040858 4255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b7916476c6b4d3daa4172f934584877 [sys.catalog]: This master's current role is: LEADER
I20260810 07:18:44.041594 4266 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:18:44.043287 4266 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:18:44.045279 4266 catalog_manager.cc:1347] Generated new cluster ID: 40dd6e9649eb45f2b47664832f728e86
I20260810 07:18:44.045342 4266 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:18:44.046654 4258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:44.040044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37652 (local address 127.15.217.126:44657)
0810 07:18:44.040278 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:44.040288 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:44.041258 (+ 970us) server_negotiation.cc:407] Connection header received
0810 07:18:44.041508 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:44.041517 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:44.041611 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:44.041798 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:44.042074 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.043089 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.044235 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.044871 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.045050 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:44.045831 (+ 781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:44.045835 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:44.045836 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:18:44.046064 (+ 228us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:44.046110 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:18:44.046125 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:44.046216 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:44.046321 (+ 105us) server_negotiation.cc:299] Negotiation successful
0810 07:18:44.046415 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":93,"threads_started":1}
I20260810 07:18:44.068535 4266 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:18:44.069168 4266 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:18:44.072388 4266 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b7916476c6b4d3daa4172f934584877: Generated new TSK 0
I20260810 07:18:44.072629 4266 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:18:44.081574 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.65:0
--local_ip_for_outbound_sockets=127.15.217.65
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:44657
--builtin_ntp_servers=127.15.217.84:41179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 07:18:44.171161 4277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:44.171386 4277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:44.171409 4277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:44.172611 4277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:44.172653 4277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:44.172679 4277 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.65
I20260810 07:18:44.174002 4277 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:41179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:44657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.65
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:44.174218 4277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:44.174510 4277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:18:44.180243 4277 server_base.cc:1034] running on GCE node
W20260810 07:18:44.180136 4283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:44.180136 4284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:44.180164 4288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:44.180789 4277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:44.181049 4277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:44.182199 4277 hybrid_clock.cc:648] HybridClock initialized: now 1786346324182174 us; error 50 us; skew 500 ppm
I20260810 07:18:44.183737 4277 webserver.cc:460] Webserver started at http://127.15.217.65:37899/ using document root <none> and password file <none>
I20260810 07:18:44.184052 4277 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:44.184114 4277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:44.184314 4277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:44.185616 4277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-0/data/instance:
uuid: "89b18ae951604c6fa84c81e05f061211"
format_stamp: "Formatted at 2026-08-10 07:18:44 on dist-test-slave-10pc"
server_key: "498869b1c32e9f3d99e33d9cba2b981a"
server_key_iv: "26057a5d58b288e99e52d25bde7466e2"
server_key_version: "encryptionkey@0"
I20260810 07:18:44.186030 4277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance:
uuid: "89b18ae951604c6fa84c81e05f061211"
format_stamp: "Formatted at 2026-08-10 07:18:44 on dist-test-slave-10pc"
server_key: "498869b1c32e9f3d99e33d9cba2b981a"
server_key_iv: "26057a5d58b288e99e52d25bde7466e2"
server_key_version: "encryptionkey@0"
I20260810 07:18:44.187592 4277 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:18:44.188534 4294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:44.188767 4277 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:18:44.188887 4277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-0/wal
uuid: "89b18ae951604c6fa84c81e05f061211"
format_stamp: "Formatted at 2026-08-10 07:18:44 on dist-test-slave-10pc"
server_key: "498869b1c32e9f3d99e33d9cba2b981a"
server_key_iv: "26057a5d58b288e99e52d25bde7466e2"
server_key_version: "encryptionkey@0"
I20260810 07:18:44.188987 4277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:44.205969 4277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:44.206382 4277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:44.206562 4277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:44.206838 4277 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:18:44.207252 4277 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:18:44.207324 4277 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:44.207374 4277 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:18:44.207401 4277 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:44.213902 4277 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.65:45137
I20260810 07:18:44.213955 4416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.65:45137 every 8 connection(s)
I20260810 07:18:44.214382 4277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
I20260810 07:18:44.218281 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 4277
I20260810 07:18:44.218420 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance
I20260810 07:18:44.218643 16229 external_mini_cluster.cc:1428] Setting key 63a2439be904b517b3c917b69001b230
I20260810 07:18:44.219712 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.66:0
--local_ip_for_outbound_sockets=127.15.217.66
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:44657
--builtin_ntp_servers=127.15.217.84:41179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 07:18:44.220839 4258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:44.215322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:57229 (local address 127.15.217.126:44657)
0810 07:18:44.215403 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:44.215410 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:44.215688 (+ 278us) server_negotiation.cc:407] Connection header received
0810 07:18:44.216191 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:44.216192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:44.216203 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:44.216268 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:44.217000 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.217499 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.218694 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.218956 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.219038 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:44.220394 (+ 1356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:44.220396 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:44.220397 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:18:44.220417 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:44.220437 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:18:44.220442 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:44.220510 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:44.220672 (+ 162us) server_negotiation.cc:299] Negotiation successful
0810 07:18:44.220718 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 07:18:44.221371 4420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:44.215310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:44657 (local address 127.15.217.65:57229)
0810 07:18:44.215528 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:44.215577 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:44.216103 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:44.216350 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:44.216353 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:44.216362 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:18:44.216919 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:44.216924 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.217595 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.217596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:44.218584 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:44.218588 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.219051 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.219052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:44.219203 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:44.220203 (+ 1000us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:18:44.220204 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:18:44.220248 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:18:44.220282 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:18:44.220562 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:18:44.220569 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:18:44.220574 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:18:44.220958 (+ 384us) client_negotiation.cc:240] Negotiation successful
0810 07:18:44.221096 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":105,"threads_started":1}
I20260810 07:18:44.222136 4418 heartbeater.cc:347] Connected to a master server at 127.15.217.126:44657
I20260810 07:18:44.222262 4418 heartbeater.cc:464] Registering TS with master...
I20260810 07:18:44.222499 4418 heartbeater.cc:511] Master 127.15.217.126:44657 requested a full tablet report, sending...
I20260810 07:18:44.223184 4210 ts_manager.cc:194] Registered new tserver with Master: 89b18ae951604c6fa84c81e05f061211 (127.15.217.65:45137)
I20260810 07:18:44.224133 4210 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.217.65:57229
W20260810 07:18:44.313580 4421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:44.313866 4421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:44.313906 4421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:44.315203 4421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:44.315282 4421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:44.315330 4421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.66
I20260810 07:18:44.316979 4421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:41179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:44657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.66
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:44.317308 4421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:44.317668 4421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:44.323189 4427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:44.323238 4426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:44.323380 4421 server_base.cc:1034] running on GCE node
W20260810 07:18:44.323207 4429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:44.323709 4421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:44.323958 4421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:44.325122 4421 hybrid_clock.cc:648] HybridClock initialized: now 1786346324324961 us; error 175 us; skew 500 ppm
I20260810 07:18:44.326506 4421 webserver.cc:460] Webserver started at http://127.15.217.66:43701/ using document root <none> and password file <none>
I20260810 07:18:44.326833 4421 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:44.326915 4421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:44.327080 4421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:44.328235 4421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-1/data/instance:
uuid: "f353c247f95e44f3afb7e81d94206186"
format_stamp: "Formatted at 2026-08-10 07:18:44 on dist-test-slave-10pc"
server_key: "ca4562145bf78b6375d40b2614157b64"
server_key_iv: "bd9a2d80bf16e0af21ee77f43f8b6eb9"
server_key_version: "encryptionkey@0"
I20260810 07:18:44.328675 4421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance:
uuid: "f353c247f95e44f3afb7e81d94206186"
format_stamp: "Formatted at 2026-08-10 07:18:44 on dist-test-slave-10pc"
server_key: "ca4562145bf78b6375d40b2614157b64"
server_key_iv: "bd9a2d80bf16e0af21ee77f43f8b6eb9"
server_key_version: "encryptionkey@0"
I20260810 07:18:44.330202 4421 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:18:44.331102 4437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:44.331310 4421 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:18:44.331430 4421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-1/wal
uuid: "f353c247f95e44f3afb7e81d94206186"
format_stamp: "Formatted at 2026-08-10 07:18:44 on dist-test-slave-10pc"
server_key: "ca4562145bf78b6375d40b2614157b64"
server_key_iv: "bd9a2d80bf16e0af21ee77f43f8b6eb9"
server_key_version: "encryptionkey@0"
I20260810 07:18:44.331517 4421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:44.352286 4421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:44.352717 4421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:44.352903 4421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:44.353174 4421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:18:44.353567 4421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:18:44.353629 4421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:44.353677 4421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:18:44.353725 4421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:44.359910 4421 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.66:46203
I20260810 07:18:44.359968 4564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.66:46203 every 8 connection(s)
I20260810 07:18:44.360353 4421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
I20260810 07:18:44.365933 4258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:44.361238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:35463 (local address 127.15.217.126:44657)
0810 07:18:44.361317 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:44.361320 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:44.361591 (+ 271us) server_negotiation.cc:407] Connection header received
0810 07:18:44.362133 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:44.362135 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:44.362146 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:44.362201 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:44.362926 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.363433 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.364529 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.364748 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.364807 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:44.365673 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:44.365675 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:44.365675 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:18:44.365680 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:44.365696 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:18:44.365700 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:44.365745 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:44.365841 (+ 96us) server_negotiation.cc:299] Negotiation successful
0810 07:18:44.365870 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:18:44.366146 4567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:44.361222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:44657 (local address 127.15.217.66:35463)
0810 07:18:44.361453 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:44.361493 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:44.362024 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:44.362268 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:44.362272 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:44.362281 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:18:44.362847 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:44.362852 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.363532 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.363532 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:44.364379 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:44.364382 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.364836 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.364836 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:44.364978 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:44.365525 (+ 547us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:18:44.365526 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:18:44.365557 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:18:44.365583 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:18:44.365762 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:18:44.365767 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:18:44.365769 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 07:18:44.365826 (+ 57us) client_negotiation.cc:240] Negotiation successful
0810 07:18:44.365914 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":116,"threads_started":1}
I20260810 07:18:44.366714 4565 heartbeater.cc:347] Connected to a master server at 127.15.217.126:44657
I20260810 07:18:44.366835 4565 heartbeater.cc:464] Registering TS with master...
I20260810 07:18:44.367046 4565 heartbeater.cc:511] Master 127.15.217.126:44657 requested a full tablet report, sending...
I20260810 07:18:44.367530 4210 ts_manager.cc:194] Registered new tserver with Master: f353c247f95e44f3afb7e81d94206186 (127.15.217.66:46203)
I20260810 07:18:44.367587 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 4421
I20260810 07:18:44.367710 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance
I20260810 07:18:44.367950 16229 external_mini_cluster.cc:1428] Setting key e06f483e71dda1495ffe210c3e3f514e
I20260810 07:18:44.368008 4210 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.217.66:35463
I20260810 07:18:44.368944 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.67:0
--local_ip_for_outbound_sockets=127.15.217.67
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:44657
--builtin_ntp_servers=127.15.217.84:41179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 07:18:44.459072 4568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:44.459321 4568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:44.459347 4568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:44.460666 4568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:44.460718 4568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:44.460745 4568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.67
I20260810 07:18:44.462239 4568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:41179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:44657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.67
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:44.462538 4568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:44.462893 4568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:44.468657 4574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:44.468695 4575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:44.468657 4577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:44.468966 4568 server_base.cc:1034] running on GCE node
I20260810 07:18:44.469213 4568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:44.469554 4568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:44.470746 4568 hybrid_clock.cc:648] HybridClock initialized: now 1786346324470718 us; error 37 us; skew 500 ppm
I20260810 07:18:44.472195 4568 webserver.cc:460] Webserver started at http://127.15.217.67:33349/ using document root <none> and password file <none>
I20260810 07:18:44.472592 4568 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:44.472675 4568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:44.472833 4568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:44.474073 4568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-2/data/instance:
uuid: "c21985e6afa04454a24a298a8cd969b5"
format_stamp: "Formatted at 2026-08-10 07:18:44 on dist-test-slave-10pc"
server_key: "c9e8d37e7759cc2b6f98713b413a9467"
server_key_iv: "a301973bbb9e83599626ef82014f36b1"
server_key_version: "encryptionkey@0"
I20260810 07:18:44.474470 4568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance:
uuid: "c21985e6afa04454a24a298a8cd969b5"
format_stamp: "Formatted at 2026-08-10 07:18:44 on dist-test-slave-10pc"
server_key: "c9e8d37e7759cc2b6f98713b413a9467"
server_key_iv: "a301973bbb9e83599626ef82014f36b1"
server_key_version: "encryptionkey@0"
I20260810 07:18:44.475930 4568 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:18:44.476775 4584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:44.476965 4568 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:18:44.477066 4568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-2/wal
uuid: "c21985e6afa04454a24a298a8cd969b5"
format_stamp: "Formatted at 2026-08-10 07:18:44 on dist-test-slave-10pc"
server_key: "c9e8d37e7759cc2b6f98713b413a9467"
server_key_iv: "a301973bbb9e83599626ef82014f36b1"
server_key_version: "encryptionkey@0"
I20260810 07:18:44.477156 4568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:44.495702 4568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:44.496063 4568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:44.496232 4568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:44.496579 4568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:18:44.497010 4568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:18:44.497071 4568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:44.497114 4568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:18:44.497140 4568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:44.503476 4568 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.67:42471
I20260810 07:18:44.503525 4711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.67:42471 every 8 connection(s)
I20260810 07:18:44.503904 4568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
I20260810 07:18:44.505300 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 4568
I20260810 07:18:44.505393 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance
I20260810 07:18:44.505625 16229 external_mini_cluster.cc:1428] Setting key e3c2f9545d73e60145b25b116b10be4d
I20260810 07:18:44.509754 4258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:44.504759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:58099 (local address 127.15.217.126:44657)
0810 07:18:44.504807 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:44.504809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:44.505096 (+ 287us) server_negotiation.cc:407] Connection header received
0810 07:18:44.505856 (+ 760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:44.505857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:44.505869 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:44.505931 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:44.506846 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.507357 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.508330 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.508600 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.508651 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:44.509541 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:44.509541 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:44.509542 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:18:44.509545 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:44.509557 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:18:44.509560 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:44.509600 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:44.509651 (+ 51us) server_negotiation.cc:299] Negotiation successful
0810 07:18:44.509685 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 07:18:44.510017 4714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:44.504738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:44657 (local address 127.15.217.67:58099)
0810 07:18:44.504926 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:44.504979 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:44.505714 (+ 735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:44.506020 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:44.506024 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:44.506032 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:18:44.506767 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:44.506775 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.507443 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.507444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:44.508248 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:44.508250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.508673 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.508674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:44.508793 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:44.509407 (+ 614us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:18:44.509407 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:18:44.509438 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:18:44.509463 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:18:44.509602 (+ 139us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:18:44.509604 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:18:44.509606 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 07:18:44.509653 (+ 47us) client_negotiation.cc:240] Negotiation successful
0810 07:18:44.509742 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":79,"threads_started":1}
I20260810 07:18:44.510473 4712 heartbeater.cc:347] Connected to a master server at 127.15.217.126:44657
I20260810 07:18:44.510603 4712 heartbeater.cc:464] Registering TS with master...
I20260810 07:18:44.510813 4712 heartbeater.cc:511] Master 127.15.217.126:44657 requested a full tablet report, sending...
I20260810 07:18:44.511260 4210 ts_manager.cc:194] Registered new tserver with Master: c21985e6afa04454a24a298a8cd969b5 (127.15.217.67:42471)
I20260810 07:18:44.511664 4210 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.217.67:58099
I20260810 07:18:44.517794 16229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:18:44.522475 4258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:44.519349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37658 (local address 127.15.217.126:44657)
0810 07:18:44.519423 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:44.519425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:44.519572 (+ 147us) server_negotiation.cc:407] Connection header received
0810 07:18:44.519641 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:44.519642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:44.519655 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:44.519705 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:44.519880 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.520402 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.521224 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.521417 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.521474 (+ 57us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:44.522135 (+ 661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:44.522136 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:44.522136 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:18:44.522141 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:44.522156 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:18:44.522161 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:44.522242 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:44.522323 (+ 81us) server_negotiation.cc:299] Negotiation successful
0810 07:18:44.522334 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 07:18:44.524312 4210 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37658:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:18:44.525974 4210 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:18:44.532766 4724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:44.528581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:46203 (local address 127.0.0.1:58470)
0810 07:18:44.528851 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:44.528862 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:44.528913 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:44.529283 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:44.529284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:44.529285 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:44.529335 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:44.529337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.530216 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.530217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:44.531490 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:44.531493 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.532398 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.532399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:44.532518 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:44.532539 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 07:18:44.532614 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 07:18:44.532684 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":77,"threads_started":1}
I20260810 07:18:44.532775 4726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:44.528831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58470 (local address 127.15.217.66:46203)
0810 07:18:44.529045 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:44.529049 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:44.529059 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:18:44.529106 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:44.529108 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:44.529125 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:44.529203 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:44.529397 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.530109 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.531573 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.532297 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.532381 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:44.532520 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:44.532631 (+ 111us) server_negotiation.cc:299] Negotiation successful
0810 07:18:44.532677 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":101,"threads_started":1}
I20260810 07:18:44.532889 4720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:44.528325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:45137 (local address 127.0.0.1:52062)
0810 07:18:44.528480 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:44.528525 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:44.528590 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:44.529129 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:44.529134 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:44.529137 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:44.529233 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:44.529236 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.530623 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.530624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:44.531681 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:44.531682 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.532661 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.532662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:44.532727 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:44.532731 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:18:44.532761 (+ 30us) client_negotiation.cc:240] Negotiation successful
0810 07:18:44.532808 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":80,"threads_started":1}
I20260810 07:18:44.533209 4725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:44.528753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52062 (local address 127.15.217.65:45137)
0810 07:18:44.528919 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:44.528924 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:44.528936 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:18:44.528984 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:44.528986 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:44.529003 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:44.529318 (+ 315us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:44.529380 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.530529 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.531766 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.532563 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.532955 (+ 392us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:44.533054 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:44.533077 (+ 23us) server_negotiation.cc:299] Negotiation successful
0810 07:18:44.533110 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":92,"threads_started":1}
I20260810 07:18:44.533924 4721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:44.528329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:42471 (local address 127.0.0.1:48188)
0810 07:18:44.528555 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:44.528567 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:44.528625 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:44.529210 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:44.529211 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:44.529212 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:44.529242 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:44.529243 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.531753 (+ 2510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.531753 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:44.532446 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:44.532448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.533464 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.533466 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:44.533535 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:44.533539 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:18:44.533600 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 07:18:44.533850 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":72,"threads_started":1}
I20260810 07:18:44.534011 4488 tablet_service.cc:1467] Processing CreateTablet for tablet 748d16125644480b94388f3c6830a2d5 (DEFAULT_TABLE table=test-table [id=7c4856e117d04f15b189fd0c96a4a115]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:18:44.534009 4346 tablet_service.cc:1467] Processing CreateTablet for tablet 748d16125644480b94388f3c6830a2d5 (DEFAULT_TABLE table=test-table [id=7c4856e117d04f15b189fd0c96a4a115]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:18:44.534439 4488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 748d16125644480b94388f3c6830a2d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:44.534439 4346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 748d16125644480b94388f3c6830a2d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:44.538318 4727 tablet_bootstrap.cc:492] T 748d16125644480b94388f3c6830a2d5 P 89b18ae951604c6fa84c81e05f061211: Bootstrap starting.
I20260810 07:18:44.538496 4723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:44.528548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48188 (local address 127.15.217.67:42471)
0810 07:18:44.529042 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:44.529046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:44.529056 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:18:44.529097 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:44.529099 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:44.529114 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:44.529315 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:44.529371 (+ 56us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.530527 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.532565 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.533366 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.533442 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:44.533525 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:44.538318 (+ 4793us) server_negotiation.cc:299] Negotiation successful
0810 07:18:44.538406 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":96,"threads_started":1}
I20260810 07:18:44.539131 4727 tablet_bootstrap.cc:654] T 748d16125644480b94388f3c6830a2d5 P 89b18ae951604c6fa84c81e05f061211: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:44.539209 4639 tablet_service.cc:1467] Processing CreateTablet for tablet 748d16125644480b94388f3c6830a2d5 (DEFAULT_TABLE table=test-table [id=7c4856e117d04f15b189fd0c96a4a115]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:18:44.539495 4727 log.cc:824] T 748d16125644480b94388f3c6830a2d5 P 89b18ae951604c6fa84c81e05f061211: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:44.539649 4639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 748d16125644480b94388f3c6830a2d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:44.539956 4728 tablet_bootstrap.cc:492] T 748d16125644480b94388f3c6830a2d5 P f353c247f95e44f3afb7e81d94206186: Bootstrap starting.
I20260810 07:18:44.540802 4728 tablet_bootstrap.cc:654] T 748d16125644480b94388f3c6830a2d5 P f353c247f95e44f3afb7e81d94206186: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:44.540910 4727 tablet_bootstrap.cc:492] T 748d16125644480b94388f3c6830a2d5 P 89b18ae951604c6fa84c81e05f061211: No bootstrap required, opened a new log
I20260810 07:18:44.541002 4727 ts_tablet_manager.cc:1397] T 748d16125644480b94388f3c6830a2d5 P 89b18ae951604c6fa84c81e05f061211: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:18:44.541193 4728 log.cc:824] T 748d16125644480b94388f3c6830a2d5 P f353c247f95e44f3afb7e81d94206186: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:44.542088 4728 tablet_bootstrap.cc:492] T 748d16125644480b94388f3c6830a2d5 P f353c247f95e44f3afb7e81d94206186: No bootstrap required, opened a new log
I20260810 07:18:44.542299 4728 ts_tablet_manager.cc:1397] T 748d16125644480b94388f3c6830a2d5 P f353c247f95e44f3afb7e81d94206186: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260810 07:18:44.543505 4731 tablet_bootstrap.cc:492] T 748d16125644480b94388f3c6830a2d5 P c21985e6afa04454a24a298a8cd969b5: Bootstrap starting.
I20260810 07:18:44.543941 4727 raft_consensus.cc:348] T 748d16125644480b94388f3c6830a2d5 P 89b18ae951604c6fa84c81e05f061211 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89b18ae951604c6fa84c81e05f061211" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 45137 } } peers { permanent_uuid: "f353c247f95e44f3afb7e81d94206186" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46203 } } peers { permanent_uuid: "c21985e6afa04454a24a298a8cd969b5" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42471 } }
I20260810 07:18:44.544122 4727 raft_consensus.cc:374] T 748d16125644480b94388f3c6830a2d5 P 89b18ae951604c6fa84c81e05f061211 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:44.544190 4727 raft_consensus.cc:729] T 748d16125644480b94388f3c6830a2d5 P 89b18ae951604c6fa84c81e05f061211 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89b18ae951604c6fa84c81e05f061211, State: Initialized, Role: FOLLOWER
I20260810 07:18:44.544256 4731 tablet_bootstrap.cc:654] T 748d16125644480b94388f3c6830a2d5 P c21985e6afa04454a24a298a8cd969b5: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:44.544390 4727 consensus_queue.cc:260] T 748d16125644480b94388f3c6830a2d5 P 89b18ae951604c6fa84c81e05f061211 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89b18ae951604c6fa84c81e05f061211" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 45137 } } peers { permanent_uuid: "f353c247f95e44f3afb7e81d94206186" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46203 } } peers { permanent_uuid: "c21985e6afa04454a24a298a8cd969b5" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42471 } }
I20260810 07:18:44.544793 4731 log.cc:824] T 748d16125644480b94388f3c6830a2d5 P c21985e6afa04454a24a298a8cd969b5: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:44.544888 4727 ts_tablet_manager.cc:1428] T 748d16125644480b94388f3c6830a2d5 P 89b18ae951604c6fa84c81e05f061211: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260810 07:18:44.544956 4418 heartbeater.cc:503] Master 127.15.217.126:44657 was elected leader, sending a full tablet report...
I20260810 07:18:44.544826 4728 raft_consensus.cc:348] T 748d16125644480b94388f3c6830a2d5 P f353c247f95e44f3afb7e81d94206186 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89b18ae951604c6fa84c81e05f061211" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 45137 } } peers { permanent_uuid: "f353c247f95e44f3afb7e81d94206186" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46203 } } peers { permanent_uuid: "c21985e6afa04454a24a298a8cd969b5" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42471 } }
I20260810 07:18:44.545380 4728 raft_consensus.cc:374] T 748d16125644480b94388f3c6830a2d5 P f353c247f95e44f3afb7e81d94206186 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:44.545444 4728 raft_consensus.cc:729] T 748d16125644480b94388f3c6830a2d5 P f353c247f95e44f3afb7e81d94206186 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f353c247f95e44f3afb7e81d94206186, State: Initialized, Role: FOLLOWER
I20260810 07:18:44.545581 4728 consensus_queue.cc:260] T 748d16125644480b94388f3c6830a2d5 P f353c247f95e44f3afb7e81d94206186 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89b18ae951604c6fa84c81e05f061211" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 45137 } } peers { permanent_uuid: "f353c247f95e44f3afb7e81d94206186" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46203 } } peers { permanent_uuid: "c21985e6afa04454a24a298a8cd969b5" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42471 } }
I20260810 07:18:44.545671 4731 tablet_bootstrap.cc:492] T 748d16125644480b94388f3c6830a2d5 P c21985e6afa04454a24a298a8cd969b5: No bootstrap required, opened a new log
I20260810 07:18:44.545751 4731 ts_tablet_manager.cc:1397] T 748d16125644480b94388f3c6830a2d5 P c21985e6afa04454a24a298a8cd969b5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:18:44.545847 4728 ts_tablet_manager.cc:1428] T 748d16125644480b94388f3c6830a2d5 P f353c247f95e44f3afb7e81d94206186: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260810 07:18:44.546119 4565 heartbeater.cc:503] Master 127.15.217.126:44657 was elected leader, sending a full tablet report...
I20260810 07:18:44.548190 4731 raft_consensus.cc:348] T 748d16125644480b94388f3c6830a2d5 P c21985e6afa04454a24a298a8cd969b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89b18ae951604c6fa84c81e05f061211" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 45137 } } peers { permanent_uuid: "f353c247f95e44f3afb7e81d94206186" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46203 } } peers { permanent_uuid: "c21985e6afa04454a24a298a8cd969b5" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42471 } }
I20260810 07:18:44.548321 4731 raft_consensus.cc:374] T 748d16125644480b94388f3c6830a2d5 P c21985e6afa04454a24a298a8cd969b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:44.548343 4731 raft_consensus.cc:729] T 748d16125644480b94388f3c6830a2d5 P c21985e6afa04454a24a298a8cd969b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c21985e6afa04454a24a298a8cd969b5, State: Initialized, Role: FOLLOWER
I20260810 07:18:44.548456 4731 consensus_queue.cc:260] T 748d16125644480b94388f3c6830a2d5 P c21985e6afa04454a24a298a8cd969b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89b18ae951604c6fa84c81e05f061211" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 45137 } } peers { permanent_uuid: "f353c247f95e44f3afb7e81d94206186" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46203 } } peers { permanent_uuid: "c21985e6afa04454a24a298a8cd969b5" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42471 } }
I20260810 07:18:44.548699 4731 ts_tablet_manager.cc:1428] T 748d16125644480b94388f3c6830a2d5 P c21985e6afa04454a24a298a8cd969b5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:18:44.548825 4712 heartbeater.cc:503] Master 127.15.217.126:44657 was elected leader, sending a full tablet report...
W20260810 07:18:44.611052 4566 tablet.cc:2369] T 748d16125644480b94388f3c6830a2d5 P f353c247f95e44f3afb7e81d94206186: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:18:44.696321 4732 raft_consensus.cc:482] T 748d16125644480b94388f3c6830a2d5 P 89b18ae951604c6fa84c81e05f061211 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:18:44.696547 4732 raft_consensus.cc:504] T 748d16125644480b94388f3c6830a2d5 P 89b18ae951604c6fa84c81e05f061211 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89b18ae951604c6fa84c81e05f061211" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 45137 } } peers { permanent_uuid: "f353c247f95e44f3afb7e81d94206186" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46203 } } peers { permanent_uuid: "c21985e6afa04454a24a298a8cd969b5" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42471 } }
I20260810 07:18:44.697050 4732 leader_election.cc:288] T 748d16125644480b94388f3c6830a2d5 P 89b18ae951604c6fa84c81e05f061211 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f353c247f95e44f3afb7e81d94206186 (127.15.217.66:46203), c21985e6afa04454a24a298a8cd969b5 (127.15.217.67:42471)
I20260810 07:18:44.700124 4420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:44.697261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:46203 (local address 127.15.217.65:44611)
0810 07:18:44.697325 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:44.697337 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:44.697389 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:44.697545 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:44.697546 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:44.697548 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:44.697643 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:44.697646 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.698447 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.698448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:44.699295 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:44.699298 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.699878 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.699880 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:44.699959 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:44.699963 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:18:44.700019 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 07:18:44.700068 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 07:18:44.700217 4726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:44.697307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:44611 (local address 127.15.217.66:46203)
0810 07:18:44.697371 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:44.697374 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:44.697388 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:18:44.697453 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:44.697454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:44.697464 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:44.697517 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:44.697779 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.698361 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.699404 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.699792 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.699847 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:44.699854 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:44.700049 (+ 195us) server_negotiation.cc:299] Negotiation successful
0810 07:18:44.700143 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 07:18:44.700278 4736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:44.697271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:42471 (local address 127.15.217.65:55451)
0810 07:18:44.697633 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:44.697643 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:44.697696 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:44.697851 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:44.697852 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:44.697854 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:44.697933 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:44.697934 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.698615 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.698616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:44.699391 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:44.699394 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.699979 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.699980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:44.700087 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:44.700092 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:18:44.700153 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 07:18:44.700190 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"mutex_wait_us":178,"thread_start_us":97,"threads_started":1}
I20260810 07:18:44.700335 4723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:44.697345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:55451 (local address 127.15.217.67:42471)
0810 07:18:44.697422 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:44.697424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:44.697724 (+ 300us) server_negotiation.cc:407] Connection header received
0810 07:18:44.697774 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:44.697775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:44.697785 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:44.697837 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:44.698007 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.698522 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.699493 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.699885 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.699960 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:44.699968 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:44.700193 (+ 225us) server_negotiation.cc:299] Negotiation successful
0810 07:18:44.700247 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 07:18:44.700711 4512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "748d16125644480b94388f3c6830a2d5" candidate_uuid: "89b18ae951604c6fa84c81e05f061211" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f353c247f95e44f3afb7e81d94206186" is_pre_election: true
I20260810 07:18:44.700786 4662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "748d16125644480b94388f3c6830a2d5" candidate_uuid: "89b18ae951604c6fa84c81e05f061211" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c21985e6afa04454a24a298a8cd969b5" is_pre_election: true
I20260810 07:18:44.700901 4512 raft_consensus.cc:2454] T 748d16125644480b94388f3c6830a2d5 P f353c247f95e44f3afb7e81d94206186 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89b18ae951604c6fa84c81e05f061211 in term 0.
I20260810 07:18:44.700907 4662 raft_consensus.cc:2454] T 748d16125644480b94388f3c6830a2d5 P c21985e6afa04454a24a298a8cd969b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89b18ae951604c6fa84c81e05f061211 in term 0.
I20260810 07:18:44.701159 4295 leader_election.cc:302] T 748d16125644480b94388f3c6830a2d5 P 89b18ae951604c6fa84c81e05f061211 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89b18ae951604c6fa84c81e05f061211, c21985e6afa04454a24a298a8cd969b5; no voters:
I20260810 07:18:44.701313 4732 raft_consensus.cc:2781] T 748d16125644480b94388f3c6830a2d5 P 89b18ae951604c6fa84c81e05f061211 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:18:44.701370 4732 raft_consensus.cc:482] T 748d16125644480b94388f3c6830a2d5 P 89b18ae951604c6fa84c81e05f061211 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:18:44.701395 4732 raft_consensus.cc:3037] T 748d16125644480b94388f3c6830a2d5 P 89b18ae951604c6fa84c81e05f061211 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:44.702201 4732 raft_consensus.cc:504] T 748d16125644480b94388f3c6830a2d5 P 89b18ae951604c6fa84c81e05f061211 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89b18ae951604c6fa84c81e05f061211" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 45137 } } peers { permanent_uuid: "f353c247f95e44f3afb7e81d94206186" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46203 } } peers { permanent_uuid: "c21985e6afa04454a24a298a8cd969b5" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42471 } }
I20260810 07:18:44.702361 4732 leader_election.cc:288] T 748d16125644480b94388f3c6830a2d5 P 89b18ae951604c6fa84c81e05f061211 [CANDIDATE]: Term 1 election: Requested vote from peers f353c247f95e44f3afb7e81d94206186 (127.15.217.66:46203), c21985e6afa04454a24a298a8cd969b5 (127.15.217.67:42471)
I20260810 07:18:44.702526 4662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "748d16125644480b94388f3c6830a2d5" candidate_uuid: "89b18ae951604c6fa84c81e05f061211" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c21985e6afa04454a24a298a8cd969b5"
I20260810 07:18:44.702543 4512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "748d16125644480b94388f3c6830a2d5" candidate_uuid: "89b18ae951604c6fa84c81e05f061211" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f353c247f95e44f3afb7e81d94206186"
I20260810 07:18:44.702615 4662 raft_consensus.cc:3037] T 748d16125644480b94388f3c6830a2d5 P c21985e6afa04454a24a298a8cd969b5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:44.702649 4512 raft_consensus.cc:3037] T 748d16125644480b94388f3c6830a2d5 P f353c247f95e44f3afb7e81d94206186 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:44.703389 4512 raft_consensus.cc:2454] T 748d16125644480b94388f3c6830a2d5 P f353c247f95e44f3afb7e81d94206186 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89b18ae951604c6fa84c81e05f061211 in term 1.
I20260810 07:18:44.703466 4662 raft_consensus.cc:2454] T 748d16125644480b94388f3c6830a2d5 P c21985e6afa04454a24a298a8cd969b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89b18ae951604c6fa84c81e05f061211 in term 1.
I20260810 07:18:44.703647 4298 leader_election.cc:302] T 748d16125644480b94388f3c6830a2d5 P 89b18ae951604c6fa84c81e05f061211 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89b18ae951604c6fa84c81e05f061211, f353c247f95e44f3afb7e81d94206186; no voters:
I20260810 07:18:44.703742 4732 raft_consensus.cc:2781] T 748d16125644480b94388f3c6830a2d5 P 89b18ae951604c6fa84c81e05f061211 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:18:44.703896 4732 raft_consensus.cc:686] T 748d16125644480b94388f3c6830a2d5 P 89b18ae951604c6fa84c81e05f061211 [term 1 LEADER]: Becoming Leader. State: Replica: 89b18ae951604c6fa84c81e05f061211, State: Running, Role: LEADER
I20260810 07:18:44.703977 4732 consensus_queue.cc:237] T 748d16125644480b94388f3c6830a2d5 P 89b18ae951604c6fa84c81e05f061211 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89b18ae951604c6fa84c81e05f061211" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 45137 } } peers { permanent_uuid: "f353c247f95e44f3afb7e81d94206186" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46203 } } peers { permanent_uuid: "c21985e6afa04454a24a298a8cd969b5" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42471 } }
I20260810 07:18:44.704988 4210 catalog_manager.cc:5495] T 748d16125644480b94388f3c6830a2d5 P 89b18ae951604c6fa84c81e05f061211 reported cstate change: term changed from 0 to 1, leader changed from <none> to 89b18ae951604c6fa84c81e05f061211 (127.15.217.65). New cstate: current_term: 1 leader_uuid: "89b18ae951604c6fa84c81e05f061211" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89b18ae951604c6fa84c81e05f061211" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 45137 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f353c247f95e44f3afb7e81d94206186" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c21985e6afa04454a24a298a8cd969b5" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42471 } health_report { overall_health: UNKNOWN } } }
W20260810 07:18:44.715224 4419 tablet.cc:2369] T 748d16125644480b94388f3c6830a2d5 P 89b18ae951604c6fa84c81e05f061211: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:18:44.715405 4725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:44.712870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52076 (local address 127.15.217.65:45137)
0810 07:18:44.712927 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:44.712929 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:44.712939 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:18:44.712981 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:44.712982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:44.713003 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:44.713070 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:18:44.713198 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.713673 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.714409 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:44.714605 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:44.714659 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:44.714858 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:18:44.714950 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:18:44.715025 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:44.715166 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 07:18:44.715177 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 07:18:44.717413 4662 raft_consensus.cc:1261] T 748d16125644480b94388f3c6830a2d5 P c21985e6afa04454a24a298a8cd969b5 [term 1 FOLLOWER]: Refusing update from remote peer 89b18ae951604c6fa84c81e05f061211: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:18:44.717427 4512 raft_consensus.cc:1261] T 748d16125644480b94388f3c6830a2d5 P f353c247f95e44f3afb7e81d94206186 [term 1 FOLLOWER]: Refusing update from remote peer 89b18ae951604c6fa84c81e05f061211: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:18:44.717859 4737 consensus_queue.cc:1035] T 748d16125644480b94388f3c6830a2d5 P 89b18ae951604c6fa84c81e05f061211 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c21985e6afa04454a24a298a8cd969b5" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 42471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:18:44.718011 4732 consensus_queue.cc:1035] T 748d16125644480b94388f3c6830a2d5 P 89b18ae951604c6fa84c81e05f061211 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f353c247f95e44f3afb7e81d94206186" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:18:44.721490 4743 mvcc.cc:205] Tried to move back new op lower bound from 7316874546039590912 to 7316874545988423680. Current Snapshot: MvccSnapshot[applied={T|T < 7316874546039590912 or (T in {7316874546039590912})}]
I20260810 07:18:44.722026 4742 mvcc.cc:205] Tried to move back new op lower bound from 7316874546039590912 to 7316874545988423680. Current Snapshot: MvccSnapshot[applied={T|T < 7316874546039590912}]
I20260810 07:18:44.723958 4209 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:18:44.724071 4209 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:18:44.724994 4209 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2b7916476c6b4d3daa4172f934584877: Sending DeleteTablet for 3 replicas of tablet 748d16125644480b94388f3c6830a2d5
I20260810 07:18:44.725407 4488 tablet_service.cc:1514] Processing DeleteTablet for tablet 748d16125644480b94388f3c6830a2d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:18:44 UTC) from {username='slave'} at 127.0.0.1:58470
I20260810 07:18:44.725436 4346 tablet_service.cc:1514] Processing DeleteTablet for tablet 748d16125644480b94388f3c6830a2d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:18:44 UTC) from {username='slave'} at 127.0.0.1:52062
I20260810 07:18:44.725499 4639 tablet_service.cc:1514] Processing DeleteTablet for tablet 748d16125644480b94388f3c6830a2d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:18:44 UTC) from {username='slave'} at 127.0.0.1:48188
I20260810 07:18:44.725700 4754 tablet_replica.cc:330] T 748d16125644480b94388f3c6830a2d5 P 89b18ae951604c6fa84c81e05f061211: stopping tablet replica
I20260810 07:18:44.725804 4755 tablet_replica.cc:330] T 748d16125644480b94388f3c6830a2d5 P c21985e6afa04454a24a298a8cd969b5: stopping tablet replica
I20260810 07:18:44.725873 4754 raft_consensus.cc:2229] T 748d16125644480b94388f3c6830a2d5 P 89b18ae951604c6fa84c81e05f061211 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:18:44.725971 4755 raft_consensus.cc:2229] T 748d16125644480b94388f3c6830a2d5 P c21985e6afa04454a24a298a8cd969b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:18:44.726038 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 4277
I20260810 07:18:44.726050 4754 raft_consensus.cc:2258] T 748d16125644480b94388f3c6830a2d5 P 89b18ae951604c6fa84c81e05f061211 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:18:44.726715 4755 raft_consensus.cc:2258] T 748d16125644480b94388f3c6830a2d5 P c21985e6afa04454a24a298a8cd969b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:18:44.727371 4755 ts_tablet_manager.cc:1905] T 748d16125644480b94388f3c6830a2d5 P c21985e6afa04454a24a298a8cd969b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:18:44.729664 4755 ts_tablet_manager.cc:1918] T 748d16125644480b94388f3c6830a2d5 P c21985e6afa04454a24a298a8cd969b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:18:44.729738 4755 log.cc:1196] T 748d16125644480b94388f3c6830a2d5 P c21985e6afa04454a24a298a8cd969b5: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-2/wal/wals/748d16125644480b94388f3c6830a2d5
I20260810 07:18:44.729962 4755 ts_tablet_manager.cc:1939] T 748d16125644480b94388f3c6830a2d5 P c21985e6afa04454a24a298a8cd969b5: Deleting consensus metadata
I20260810 07:18:44.730381 4753 tablet_replica.cc:330] T 748d16125644480b94388f3c6830a2d5 P f353c247f95e44f3afb7e81d94206186: stopping tablet replica
I20260810 07:18:44.730540 4753 raft_consensus.cc:2229] T 748d16125644480b94388f3c6830a2d5 P f353c247f95e44f3afb7e81d94206186 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:18:44.730696 4753 raft_consensus.cc:2258] T 748d16125644480b94388f3c6830a2d5 P f353c247f95e44f3afb7e81d94206186 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:18:44.730806 4194 catalog_manager.cc:4841] TS c21985e6afa04454a24a298a8cd969b5 (127.15.217.67:42471): tablet 748d16125644480b94388f3c6830a2d5 (table test-table [id=7c4856e117d04f15b189fd0c96a4a115]) successfully deleted
I20260810 07:18:44.731144 4753 ts_tablet_manager.cc:1905] T 748d16125644480b94388f3c6830a2d5 P f353c247f95e44f3afb7e81d94206186: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:18:44.732606 4753 ts_tablet_manager.cc:1918] T 748d16125644480b94388f3c6830a2d5 P f353c247f95e44f3afb7e81d94206186: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:18:44.732703 4753 log.cc:1196] T 748d16125644480b94388f3c6830a2d5 P f353c247f95e44f3afb7e81d94206186: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346081759552-16229-0/minicluster-data/ts-1/wal/wals/748d16125644480b94388f3c6830a2d5
I20260810 07:18:44.732934 4753 ts_tablet_manager.cc:1939] T 748d16125644480b94388f3c6830a2d5 P f353c247f95e44f3afb7e81d94206186: Deleting consensus metadata
I20260810 07:18:44.733390 4197 catalog_manager.cc:4841] TS f353c247f95e44f3afb7e81d94206186 (127.15.217.66:46203): tablet 748d16125644480b94388f3c6830a2d5 (table test-table [id=7c4856e117d04f15b189fd0c96a4a115]) successfully deleted
W20260810 07:18:44.736320 4197 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.217.65:45137 (error 108)
I20260810 07:18:44.736634 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 4421
I20260810 07:18:44.736975 4724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:44.736822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:45137 (local address 127.0.0.1:52088)
0810 07:18:44.736883 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:44.736931 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.65:45137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260810 07:18:44.738492 4197 catalog_manager.cc:4584] TS 89b18ae951604c6fa84c81e05f061211 (127.15.217.65:45137): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 748d16125644480b94388f3c6830a2d5: Network error: Client connection negotiation failed: client connection to 127.15.217.65:45137: connect: Connection refused (error 111)
I20260810 07:18:44.745311 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 4568
I20260810 07:18:44.754916 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 4179
2026-08-10T07:18:44Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (894 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:18:44 dist-test-slave-10pc krb5kdc[4759](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:18:44 dist-test-slave-10pc krb5kdc[4759](info): set up 2 sockets
Aug 10 07:18:44 dist-test-slave-10pc krb5kdc[4759](info): commencing operation
krb5kdc: starting...
W20260810 07:18:45.880429 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:18:45 dist-test-slave-10pc krb5kdc[4759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:18:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:18:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
I20260810 07:18:46.024137 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:46095
--webserver_interface=127.15.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:32833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:46095
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:18:46.112120 4776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:46.112406 4776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:46.112469 4776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:46.113670 4776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:18:46.113754 4776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:46.113776 4776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:46.113796 4776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:18:46.113829 4776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:18:46.115422 4776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:32833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:46095
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:46095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:46.115751 4776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:46.116012 4776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:46.121273 4784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:46.121284 4783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:46.121284 4786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:46.121532 4776 server_base.cc:1034] running on GCE node
I20260810 07:18:46.121917 4776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:46.122246 4776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:46.123430 4776 hybrid_clock.cc:648] HybridClock initialized: now 1786346326123414 us; error 48 us; skew 500 ppm
Aug 10 07:18:46 dist-test-slave-10pc krb5kdc[4759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346326, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:46.126411 4776 init.cc:377] Logged in from keytab as kudu/127.15.217.126@KRBTEST.COM (short username kudu)
I20260810 07:18:46.126976 4776 webserver.cc:460] Webserver started at http://127.15.217.126:42767/ using document root <none> and password file <none>
I20260810 07:18:46.127250 4776 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:46.127327 4776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:46.127489 4776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:46.128741 4776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "4c3e5c8a2cf44a198738bb7325152992"
format_stamp: "Formatted at 2026-08-10 07:18:46 on dist-test-slave-10pc"
server_key: "ac5123a6a7a3ca71c117617efc5977b4"
server_key_iv: "d2d29c5ce5ae03561f9a1d67778605d6"
server_key_version: "encryptionkey@0"
I20260810 07:18:46.129139 4776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "4c3e5c8a2cf44a198738bb7325152992"
format_stamp: "Formatted at 2026-08-10 07:18:46 on dist-test-slave-10pc"
server_key: "ac5123a6a7a3ca71c117617efc5977b4"
server_key_iv: "d2d29c5ce5ae03561f9a1d67778605d6"
server_key_version: "encryptionkey@0"
I20260810 07:18:46.130607 4776 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:18:46.131500 4793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:46.131728 4776 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:18:46.131856 4776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "4c3e5c8a2cf44a198738bb7325152992"
format_stamp: "Formatted at 2026-08-10 07:18:46 on dist-test-slave-10pc"
server_key: "ac5123a6a7a3ca71c117617efc5977b4"
server_key_iv: "d2d29c5ce5ae03561f9a1d67778605d6"
server_key_version: "encryptionkey@0"
I20260810 07:18:46.131945 4776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:46.159642 4776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:46.162449 4776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:46.162658 4776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:46.167374 4776 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:46095
I20260810 07:18:46.167414 4849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:46095 every 8 connection(s)
I20260810 07:18:46.167846 4776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:18:46.169040 4850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:46.171241 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 4776
I20260810 07:18:46.171404 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:18:46.171625 16229 external_mini_cluster.cc:1428] Setting key 867b098c8d89e05beb3d4b54d6735d9e
I20260810 07:18:46.171855 4850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c3e5c8a2cf44a198738bb7325152992: Bootstrap starting.
I20260810 07:18:46.172780 4850 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c3e5c8a2cf44a198738bb7325152992: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:46.173173 4850 log.cc:824] T 00000000000000000000000000000000 P 4c3e5c8a2cf44a198738bb7325152992: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:46.173995 4850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c3e5c8a2cf44a198738bb7325152992: No bootstrap required, opened a new log
I20260810 07:18:46.176148 4850 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c3e5c8a2cf44a198738bb7325152992 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c3e5c8a2cf44a198738bb7325152992" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 46095 } }
I20260810 07:18:46.176350 4850 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c3e5c8a2cf44a198738bb7325152992 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:46.176403 4850 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c3e5c8a2cf44a198738bb7325152992 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c3e5c8a2cf44a198738bb7325152992, State: Initialized, Role: FOLLOWER
I20260810 07:18:46.176601 4850 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c3e5c8a2cf44a198738bb7325152992 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c3e5c8a2cf44a198738bb7325152992" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 46095 } }
I20260810 07:18:46.176705 4850 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c3e5c8a2cf44a198738bb7325152992 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:18:46.176753 4850 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c3e5c8a2cf44a198738bb7325152992 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:18:46.176810 4850 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c3e5c8a2cf44a198738bb7325152992 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:46.177546 4850 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c3e5c8a2cf44a198738bb7325152992 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c3e5c8a2cf44a198738bb7325152992" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 46095 } }
I20260810 07:18:46.177682 4850 leader_election.cc:302] T 00000000000000000000000000000000 P 4c3e5c8a2cf44a198738bb7325152992 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c3e5c8a2cf44a198738bb7325152992; no voters:
I20260810 07:18:46.177902 4850 leader_election.cc:288] T 00000000000000000000000000000000 P 4c3e5c8a2cf44a198738bb7325152992 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:18:46.177959 4855 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c3e5c8a2cf44a198738bb7325152992 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:18:46.178165 4855 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c3e5c8a2cf44a198738bb7325152992 [term 1 LEADER]: Becoming Leader. State: Replica: 4c3e5c8a2cf44a198738bb7325152992, State: Running, Role: LEADER
I20260810 07:18:46.178262 4850 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c3e5c8a2cf44a198738bb7325152992 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:18:46.178356 4855 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c3e5c8a2cf44a198738bb7325152992 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c3e5c8a2cf44a198738bb7325152992" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 46095 } }
I20260810 07:18:46.178869 4855 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c3e5c8a2cf44a198738bb7325152992 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c3e5c8a2cf44a198738bb7325152992. Latest consensus state: current_term: 1 leader_uuid: "4c3e5c8a2cf44a198738bb7325152992" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c3e5c8a2cf44a198738bb7325152992" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 46095 } } }
I20260810 07:18:46.178953 4855 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c3e5c8a2cf44a198738bb7325152992 [sys.catalog]: This master's current role is: LEADER
I20260810 07:18:46.179093 4857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c3e5c8a2cf44a198738bb7325152992 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c3e5c8a2cf44a198738bb7325152992" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c3e5c8a2cf44a198738bb7325152992" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 46095 } } }
I20260810 07:18:46.179185 4857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c3e5c8a2cf44a198738bb7325152992 [sys.catalog]: This master's current role is: LEADER
Aug 10 07:18:46 dist-test-slave-10pc krb5kdc[4759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:18:46.180182 4863 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:18:46.181170 4863 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:18:46.183195 4863 catalog_manager.cc:1347] Generated new cluster ID: 92d07c1f5a5d4d4e909a4d35346e2a18
I20260810 07:18:46.183252 4863 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:18:46.184530 4853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:46.172298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38808 (local address 127.15.217.126:46095)
0810 07:18:46.172531 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:46.172542 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:46.172614 (+ 72us) server_negotiation.cc:407] Connection header received
0810 07:18:46.173472 (+ 858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:46.173480 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:46.173549 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:46.173734 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:46.174523 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:46.175513 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:46.176342 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:46.177071 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:46.177215 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:46.180439 (+ 3224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:46.180444 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:46.180445 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:46.180482 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:46.182587 (+ 2105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:46.183058 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:46.183062 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:46.183062 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:46.183181 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:46.183449 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:46.183450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:46.183450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:46.183776 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:46.183934 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:46.184120 (+ 186us) server_negotiation.cc:299] Negotiation successful
0810 07:18:46.184280 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":109,"threads_started":1}
I20260810 07:18:46.204918 4863 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:18:46.205533 4863 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:18:46.210603 4863 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c3e5c8a2cf44a198738bb7325152992: Generated new TSK 0
I20260810 07:18:46.210862 4863 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.217.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab.
Entry for principal kudu/127.15.217.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab.
I20260810 07:18:46.249264 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.65:0
--local_ip_for_outbound_sockets=127.15.217.65
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:46095
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:32833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:18:46.339933 4878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:46.340190 4878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:46.340219 4878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:46.341566 4878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:46.341624 4878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:46.341653 4878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.65
I20260810 07:18:46.343375 4878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:32833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.65.keytab
--principal=kudu/127.15.217.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:46095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.65
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:46.343753 4878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:46.344061 4878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:46.349715 4883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:46.349725 4884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:46.349937 4878 server_base.cc:1034] running on GCE node
W20260810 07:18:46.349725 4886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:46.350318 4878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:46.350616 4878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:46.351816 4878 hybrid_clock.cc:648] HybridClock initialized: now 1786346326351804 us; error 31 us; skew 500 ppm
Aug 10 07:18:46 dist-test-slave-10pc krb5kdc[4759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346326, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:46.354815 4878 init.cc:377] Logged in from keytab as kudu/127.15.217.65@KRBTEST.COM (short username kudu)
I20260810 07:18:46.355294 4878 webserver.cc:460] Webserver started at http://127.15.217.65:35989/ using document root <none> and password file <none>
I20260810 07:18:46.355491 4878 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:46.355542 4878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:46.355688 4878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:46.356926 4878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-0/data/instance:
uuid: "1a2ecd3c1ea24075a665a1b393160c8d"
format_stamp: "Formatted at 2026-08-10 07:18:46 on dist-test-slave-10pc"
server_key: "3d48ed7fd1c29e09ea5d02866ec3f2e8"
server_key_iv: "3349ff52eb0c0edcdeb3d9942f2931cc"
server_key_version: "encryptionkey@0"
I20260810 07:18:46.357259 4878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance:
uuid: "1a2ecd3c1ea24075a665a1b393160c8d"
format_stamp: "Formatted at 2026-08-10 07:18:46 on dist-test-slave-10pc"
server_key: "3d48ed7fd1c29e09ea5d02866ec3f2e8"
server_key_iv: "3349ff52eb0c0edcdeb3d9942f2931cc"
server_key_version: "encryptionkey@0"
I20260810 07:18:46.358714 4878 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:18:46.359558 4894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:46.359800 4878 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:18:46.359925 4878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-0/wal
uuid: "1a2ecd3c1ea24075a665a1b393160c8d"
format_stamp: "Formatted at 2026-08-10 07:18:46 on dist-test-slave-10pc"
server_key: "3d48ed7fd1c29e09ea5d02866ec3f2e8"
server_key_iv: "3349ff52eb0c0edcdeb3d9942f2931cc"
server_key_version: "encryptionkey@0"
I20260810 07:18:46.360018 4878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:46.372263 4878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:46.375113 4878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:46.375332 4878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:46.375631 4878 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:18:46.376096 4878 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:18:46.376160 4878 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:46.376197 4878 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:18:46.376224 4878 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:46.383074 4878 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.65:40181
I20260810 07:18:46.383118 5015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.65:40181 every 8 connection(s)
I20260810 07:18:46.383524 4878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
I20260810 07:18:46.386256 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 4878
I20260810 07:18:46.386407 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance
I20260810 07:18:46.386660 16229 external_mini_cluster.cc:1428] Setting key 1762c755fbe8b423c07728ac44e9d8c2
Aug 10 07:18:46 dist-test-slave-10pc krb5kdc[4759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346326, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.65@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:18:46.394573 4853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:46.384397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:59015 (local address 127.15.217.126:46095)
0810 07:18:46.384584 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:46.384586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:46.384731 (+ 145us) server_negotiation.cc:407] Connection header received
0810 07:18:46.385301 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:46.385303 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:46.385314 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:46.385383 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:46.386510 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:46.387259 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:46.388475 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:46.388726 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:46.388815 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:46.391304 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:46.391306 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:46.391306 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:46.391314 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:46.392956 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:46.393452 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:46.393454 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:46.393454 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:46.393529 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:46.393820 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:46.393821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:46.393821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:46.393955 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:46.394040 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:46.394358 (+ 318us) server_negotiation.cc:299] Negotiation successful
0810 07:18:46.394468 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157}
I20260810 07:18:46.394776 5019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:46.384307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:46095 (local address 127.15.217.65:59015)
0810 07:18:46.384563 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:46.384609 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:46.385193 (+ 584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:46.385481 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:46.385485 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:46.386127 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:18:46.386348 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:46.386353 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:46.387402 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:46.387403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:46.388343 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:46.388346 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:46.388830 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:46.388831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:46.388973 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:46.389591 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:18:46.389602 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:18:46.391189 (+ 1587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:18:46.393083 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:46.393093 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:46.393096 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:46.393357 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:46.393608 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:46.393609 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:46.393610 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:46.393744 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:46.394034 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:18:46.394042 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:18:46.394202 (+ 160us) client_negotiation.cc:769] Sending connection context
0810 07:18:46.394319 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 07:18:46.394507 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":135,"threads_started":1}
I20260810 07:18:46.395502 5017 heartbeater.cc:347] Connected to a master server at 127.15.217.126:46095
I20260810 07:18:46.395614 5017 heartbeater.cc:464] Registering TS with master...
I20260810 07:18:46.395855 5017 heartbeater.cc:511] Master 127.15.217.126:46095 requested a full tablet report, sending...
I20260810 07:18:46.396545 4811 ts_manager.cc:194] Registered new tserver with Master: 1a2ecd3c1ea24075a665a1b393160c8d (127.15.217.65:40181)
WARNING: no policy specified for kudu/127.15.217.66@KRBTEST.COM; defaulting to no policy
I20260810 07:18:46.397286 4811 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.65@KRBTEST.COM'} at 127.15.217.65:59015
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab.
Entry for principal kudu/127.15.217.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab.
I20260810 07:18:46.419106 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.66:0
--local_ip_for_outbound_sockets=127.15.217.66
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:46095
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:32833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:18:46.508317 5023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:46.508646 5023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:46.508694 5023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:46.509869 5023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:46.509941 5023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:46.509985 5023 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.66
I20260810 07:18:46.511515 5023 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:32833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.66.keytab
--principal=kudu/127.15.217.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:46095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.66
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:46.511812 5023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:46.512089 5023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:46.517644 5029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:46.517632 5031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:46.517828 5023 server_base.cc:1034] running on GCE node
W20260810 07:18:46.517937 5028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:46.518239 5023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:46.518515 5023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:46.519698 5023 hybrid_clock.cc:648] HybridClock initialized: now 1786346326519677 us; error 42 us; skew 500 ppm
Aug 10 07:18:46 dist-test-slave-10pc krb5kdc[4759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346326, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:46.522547 5023 init.cc:377] Logged in from keytab as kudu/127.15.217.66@KRBTEST.COM (short username kudu)
I20260810 07:18:46.523087 5023 webserver.cc:460] Webserver started at http://127.15.217.66:40867/ using document root <none> and password file <none>
I20260810 07:18:46.523356 5023 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:46.523433 5023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:46.523597 5023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:46.524804 5023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-1/data/instance:
uuid: "f46685e09a094f8195ad61413d461d6c"
format_stamp: "Formatted at 2026-08-10 07:18:46 on dist-test-slave-10pc"
server_key: "774ded1130c3d462e9052a00fb922514"
server_key_iv: "8405d8fd691fdb6f028f8bc046171340"
server_key_version: "encryptionkey@0"
I20260810 07:18:46.525241 5023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance:
uuid: "f46685e09a094f8195ad61413d461d6c"
format_stamp: "Formatted at 2026-08-10 07:18:46 on dist-test-slave-10pc"
server_key: "774ded1130c3d462e9052a00fb922514"
server_key_iv: "8405d8fd691fdb6f028f8bc046171340"
server_key_version: "encryptionkey@0"
I20260810 07:18:46.526803 5023 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:18:46.527676 5039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:46.527921 5023 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:18:46.528017 5023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-1/wal
uuid: "f46685e09a094f8195ad61413d461d6c"
format_stamp: "Formatted at 2026-08-10 07:18:46 on dist-test-slave-10pc"
server_key: "774ded1130c3d462e9052a00fb922514"
server_key_iv: "8405d8fd691fdb6f028f8bc046171340"
server_key_version: "encryptionkey@0"
I20260810 07:18:46.528079 5023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:46.565788 5023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:46.568558 5023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:46.568697 5023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:46.568912 5023 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:18:46.569298 5023 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:18:46.569339 5023 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:46.569363 5023 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:18:46.569417 5023 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:46.576171 5023 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.66:33963
I20260810 07:18:46.576293 5159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.66:33963 every 8 connection(s)
I20260810 07:18:46.576704 5023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
Aug 10 07:18:46 dist-test-slave-10pc krb5kdc[4759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346326, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.66@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:18:46.586357 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 5023
I20260810 07:18:46.586490 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance
I20260810 07:18:46.586753 16229 external_mini_cluster.cc:1428] Setting key 5d67c73b1ae9fe48c32f002ad1b80f3e
I20260810 07:18:46.587805 4853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:46.577572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:60637 (local address 127.15.217.126:46095)
0810 07:18:46.577641 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:46.577643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:46.577865 (+ 222us) server_negotiation.cc:407] Connection header received
0810 07:18:46.578583 (+ 718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:46.578584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:46.578594 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:46.578660 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:46.580124 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:46.580799 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:46.581902 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:46.582089 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:46.582142 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:46.584498 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:46.584500 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:46.584500 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:46.584514 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:46.586094 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:46.586651 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:46.586652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:46.586652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:46.586722 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:46.587050 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:46.587051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:46.587051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:46.587229 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:46.587300 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:46.587627 (+ 327us) server_negotiation.cc:299] Negotiation successful
0810 07:18:46.587732 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 07:18:46.588126 5163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:46.577531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:46095 (local address 127.15.217.66:60637)
0810 07:18:46.577733 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:46.577782 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:46.578469 (+ 687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:46.578737 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:46.578741 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:46.579741 (+ 1000us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:18:46.580016 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:46.580023 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:46.580896 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:46.580898 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:46.581820 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:46.581823 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:46.582174 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:46.582175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:46.582298 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:46.582813 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:18:46.582822 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:18:46.584314 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:18:46.586199 (+ 1885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:46.586209 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:46.586213 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:46.586567 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:46.586804 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:46.586805 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:46.586805 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:46.586965 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:46.587302 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:18:46.587309 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:18:46.587466 (+ 157us) client_negotiation.cc:769] Sending connection context
0810 07:18:46.587590 (+ 124us) client_negotiation.cc:240] Negotiation successful
0810 07:18:46.587794 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":117,"threads_started":1}
I20260810 07:18:46.588691 5161 heartbeater.cc:347] Connected to a master server at 127.15.217.126:46095
I20260810 07:18:46.588811 5161 heartbeater.cc:464] Registering TS with master...
I20260810 07:18:46.589022 5161 heartbeater.cc:511] Master 127.15.217.126:46095 requested a full tablet report, sending...
I20260810 07:18:46.589474 4811 ts_manager.cc:194] Registered new tserver with Master: f46685e09a094f8195ad61413d461d6c (127.15.217.66:33963)
I20260810 07:18:46.589897 4811 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.66@KRBTEST.COM'} at 127.15.217.66:60637
WARNING: no policy specified for kudu/127.15.217.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab.
Entry for principal kudu/127.15.217.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab.
I20260810 07:18:46.616540 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.67:0
--local_ip_for_outbound_sockets=127.15.217.67
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:46095
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:32833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:18:46.705402 5166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:46.705616 5166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:46.705641 5166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:46.706741 5166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:46.706779 5166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:46.706804 5166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.67
I20260810 07:18:46.708215 5166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:32833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.67.keytab
--principal=kudu/127.15.217.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:46095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.67
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:46.708422 5166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:46.708693 5166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:46.714092 5171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:46.714125 5172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:46.714092 5174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:46.714396 5166 server_base.cc:1034] running on GCE node
I20260810 07:18:46.714618 5166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:46.714885 5166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:46.716061 5166 hybrid_clock.cc:648] HybridClock initialized: now 1786346326716048 us; error 31 us; skew 500 ppm
Aug 10 07:18:46 dist-test-slave-10pc krb5kdc[4759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346326, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:46.719280 5166 init.cc:377] Logged in from keytab as kudu/127.15.217.67@KRBTEST.COM (short username kudu)
I20260810 07:18:46.719794 5166 webserver.cc:460] Webserver started at http://127.15.217.67:46061/ using document root <none> and password file <none>
I20260810 07:18:46.720083 5166 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:46.720165 5166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:46.720332 5166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:46.721531 5166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-2/data/instance:
uuid: "aad136faefcb429596c6e295a69eb365"
format_stamp: "Formatted at 2026-08-10 07:18:46 on dist-test-slave-10pc"
server_key: "0fc741da3c69e9b4e54505079a7598de"
server_key_iv: "baac6280739326ddb8578ba011f8cad7"
server_key_version: "encryptionkey@0"
I20260810 07:18:46.721942 5166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance:
uuid: "aad136faefcb429596c6e295a69eb365"
format_stamp: "Formatted at 2026-08-10 07:18:46 on dist-test-slave-10pc"
server_key: "0fc741da3c69e9b4e54505079a7598de"
server_key_iv: "baac6280739326ddb8578ba011f8cad7"
server_key_version: "encryptionkey@0"
I20260810 07:18:46.723503 5166 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:18:46.724414 5182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:46.724633 5166 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:18:46.724751 5166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-2/wal
uuid: "aad136faefcb429596c6e295a69eb365"
format_stamp: "Formatted at 2026-08-10 07:18:46 on dist-test-slave-10pc"
server_key: "0fc741da3c69e9b4e54505079a7598de"
server_key_iv: "baac6280739326ddb8578ba011f8cad7"
server_key_version: "encryptionkey@0"
I20260810 07:18:46.724843 5166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:46.738253 5166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:46.740320 5166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:46.740603 5166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:46.740885 5166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:18:46.741248 5166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:18:46.741309 5166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:46.741351 5166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:18:46.741381 5166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:46.747670 5166 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.67:38833
I20260810 07:18:46.747722 5308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.67:38833 every 8 connection(s)
I20260810 07:18:46.748087 5166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
I20260810 07:18:46.752992 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 5166
I20260810 07:18:46.753087 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance
I20260810 07:18:46.753304 16229 external_mini_cluster.cc:1428] Setting key 25ed6bf01643c39ecf6f2f2db05fb2f4
Aug 10 07:18:46 dist-test-slave-10pc krb5kdc[4759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346326, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.67@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:18:46.758262 4853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:46.748956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:55799 (local address 127.15.217.126:46095)
0810 07:18:46.749032 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:46.749034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:46.749244 (+ 210us) server_negotiation.cc:407] Connection header received
0810 07:18:46.749786 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:46.749787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:46.749798 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:46.749859 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:46.750839 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:46.751394 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:46.752504 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:46.752718 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:46.752814 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:46.755295 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:46.755297 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:46.755297 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:46.755305 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:46.756720 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:46.757158 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:46.757160 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:46.757160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:46.757225 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:46.757559 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:46.757560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:46.757561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:46.757701 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:46.757773 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:46.758113 (+ 340us) server_negotiation.cc:299] Negotiation successful
0810 07:18:46.758192 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 07:18:46.758562 5311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:46.748927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:46095 (local address 127.15.217.67:55799)
0810 07:18:46.749095 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:46.749140 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:46.749686 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:46.749919 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:46.749922 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:46.750527 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:18:46.750740 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:46.750746 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:46.751497 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:46.751498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:46.752366 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:46.752368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:46.752810 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:46.752811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:46.752927 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:46.753597 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:18:46.753607 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:18:46.755185 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:18:46.756800 (+ 1615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:46.756808 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:46.756810 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:46.757061 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:46.757325 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:46.757326 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:46.757326 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:46.757456 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:46.757789 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:18:46.757797 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:18:46.757938 (+ 141us) client_negotiation.cc:769] Sending connection context
0810 07:18:46.758066 (+ 128us) client_negotiation.cc:240] Negotiation successful
0810 07:18:46.758206 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
I20260810 07:18:46.759189 5309 heartbeater.cc:347] Connected to a master server at 127.15.217.126:46095
I20260810 07:18:46.759300 5309 heartbeater.cc:464] Registering TS with master...
I20260810 07:18:46.759523 5309 heartbeater.cc:511] Master 127.15.217.126:46095 requested a full tablet report, sending...
I20260810 07:18:46.760008 4811 ts_manager.cc:194] Registered new tserver with Master: aad136faefcb429596c6e295a69eb365 (127.15.217.67:38833)
I20260810 07:18:46.760429 4811 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.217.67@KRBTEST.COM'} at 127.15.217.67:55799
I20260810 07:18:46.765290 16229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:18:46.773068 4853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:46.766699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53992 (local address 127.15.217.126:46095)
0810 07:18:46.766770 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:46.766771 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:46.766890 (+ 119us) server_negotiation.cc:407] Connection header received
0810 07:18:46.766934 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:46.766935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:46.766944 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:46.766993 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:46.767549 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:46.768038 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:46.768917 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:46.769135 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:46.769202 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:46.770489 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:46.770490 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:46.770490 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:46.770497 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:46.771963 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:46.772286 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:46.772287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:46.772287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:46.772341 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:46.772590 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:46.772590 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:46.772591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:46.772746 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:46.772827 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:46.772930 (+ 103us) server_negotiation.cc:299] Negotiation successful
0810 07:18:46.772996 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 07:18:46.774847 4811 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53992:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:18:46.776602 4811 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:18:46.784132 5324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:46.779731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:40181 (local address 127.0.0.1:51108)
0810 07:18:46.780168 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:46.780179 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:46.780232 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:46.780636 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:46.780637 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:46.780639 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:46.780691 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:46.780693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:46.781766 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:46.781767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:46.782899 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:46.782902 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:46.783773 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:46.783774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:46.783858 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:46.783875 (+ 17us) client_negotiation.cc:769] Sending connection context
0810 07:18:46.783964 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 07:18:46.784035 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"mutex_wait_us":48,"thread_start_us":71,"threads_started":1}
I20260810 07:18:46.784137 5326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:46.779890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51108 (local address 127.15.217.65:40181)
0810 07:18:46.780421 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:46.780426 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:46.780436 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:18:46.780512 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:46.780515 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:46.780531 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:46.780616 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:46.780749 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:46.781654 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:46.783002 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:46.783665 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:46.783743 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:46.783802 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:46.783994 (+ 192us) server_negotiation.cc:299] Negotiation successful
0810 07:18:46.784039 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":120,"threads_started":1}
I20260810 07:18:46.784483 5322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:46.779513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:33963 (local address 127.0.0.1:35642)
0810 07:18:46.779726 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:46.779775 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:46.779852 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:46.780316 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:46.780320 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:46.780322 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:46.780406 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:46.780409 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:46.781908 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:46.781909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:46.783239 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:46.783242 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:46.784204 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:46.784205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:46.784278 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:46.784283 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:18:46.784322 (+ 39us) client_negotiation.cc:240] Negotiation successful
0810 07:18:46.784373 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":66,"thread_start_us":58,"threads_started":1}
I20260810 07:18:46.784576 5323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:46.779661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44020 (local address 127.15.217.67:38833)
0810 07:18:46.780359 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:46.780364 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:46.780374 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:18:46.780414 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:46.780416 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:46.780430 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:46.780537 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:46.780795 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:46.781901 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:46.783347 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:46.783948 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:46.784021 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:46.784078 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:46.784421 (+ 343us) server_negotiation.cc:299] Negotiation successful
0810 07:18:46.784499 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":100,"threads_started":1}
I20260810 07:18:46.784381 5321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:46.779559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:38833 (local address 127.0.0.1:44020)
0810 07:18:46.780135 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:46.780147 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:46.780197 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:46.780633 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:46.780634 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:46.780636 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:46.780708 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:46.780709 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:46.782026 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:46.782027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:46.783239 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:46.783242 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:46.784114 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:46.784115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:46.784180 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:46.784195 (+ 15us) client_negotiation.cc:769] Sending connection context
0810 07:18:46.784251 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 07:18:46.784312 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"mutex_wait_us":68,"thread_start_us":106,"threads_started":1}
I20260810 07:18:46.785283 5325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:46.779896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35642 (local address 127.15.217.66:33963)
0810 07:18:46.780102 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:46.780108 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:46.780120 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:18:46.780168 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:46.780170 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:46.780189 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:46.780306 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:46.780518 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:46.781779 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:46.783345 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:46.784088 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:46.784586 (+ 498us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:46.785145 (+ 559us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:46.785172 (+ 27us) server_negotiation.cc:299] Negotiation successful
0810 07:18:46.785220 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":102,"threads_started":1}
I20260810 07:18:46.785683 4947 tablet_service.cc:1467] Processing CreateTablet for tablet b33807f9eb084b12b2c2064f24a70ef3 (DEFAULT_TABLE table=test-table [id=a084a9abda38416d84108fd131df879e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:18:46.785933 5239 tablet_service.cc:1467] Processing CreateTablet for tablet b33807f9eb084b12b2c2064f24a70ef3 (DEFAULT_TABLE table=test-table [id=a084a9abda38416d84108fd131df879e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:18:46.785995 5091 tablet_service.cc:1467] Processing CreateTablet for tablet b33807f9eb084b12b2c2064f24a70ef3 (DEFAULT_TABLE table=test-table [id=a084a9abda38416d84108fd131df879e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:18:46.786111 4947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b33807f9eb084b12b2c2064f24a70ef3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:46.786301 5239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b33807f9eb084b12b2c2064f24a70ef3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:46.786351 5091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b33807f9eb084b12b2c2064f24a70ef3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:46.789201 5327 tablet_bootstrap.cc:492] T b33807f9eb084b12b2c2064f24a70ef3 P aad136faefcb429596c6e295a69eb365: Bootstrap starting.
I20260810 07:18:46.789734 5328 tablet_bootstrap.cc:492] T b33807f9eb084b12b2c2064f24a70ef3 P f46685e09a094f8195ad61413d461d6c: Bootstrap starting.
I20260810 07:18:46.789928 5327 tablet_bootstrap.cc:654] T b33807f9eb084b12b2c2064f24a70ef3 P aad136faefcb429596c6e295a69eb365: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:46.790457 5328 tablet_bootstrap.cc:654] T b33807f9eb084b12b2c2064f24a70ef3 P f46685e09a094f8195ad61413d461d6c: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:46.790537 5327 log.cc:824] T b33807f9eb084b12b2c2064f24a70ef3 P aad136faefcb429596c6e295a69eb365: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:46.791018 5328 log.cc:824] T b33807f9eb084b12b2c2064f24a70ef3 P f46685e09a094f8195ad61413d461d6c: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:46.791395 5329 tablet_bootstrap.cc:492] T b33807f9eb084b12b2c2064f24a70ef3 P 1a2ecd3c1ea24075a665a1b393160c8d: Bootstrap starting.
I20260810 07:18:46.791507 5327 tablet_bootstrap.cc:492] T b33807f9eb084b12b2c2064f24a70ef3 P aad136faefcb429596c6e295a69eb365: No bootstrap required, opened a new log
I20260810 07:18:46.791579 5327 ts_tablet_manager.cc:1397] T b33807f9eb084b12b2c2064f24a70ef3 P aad136faefcb429596c6e295a69eb365: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:18:46.791785 5328 tablet_bootstrap.cc:492] T b33807f9eb084b12b2c2064f24a70ef3 P f46685e09a094f8195ad61413d461d6c: No bootstrap required, opened a new log
I20260810 07:18:46.791877 5328 ts_tablet_manager.cc:1397] T b33807f9eb084b12b2c2064f24a70ef3 P f46685e09a094f8195ad61413d461d6c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:18:46.792169 5329 tablet_bootstrap.cc:654] T b33807f9eb084b12b2c2064f24a70ef3 P 1a2ecd3c1ea24075a665a1b393160c8d: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:46.792877 5329 log.cc:824] T b33807f9eb084b12b2c2064f24a70ef3 P 1a2ecd3c1ea24075a665a1b393160c8d: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:46.793758 5329 tablet_bootstrap.cc:492] T b33807f9eb084b12b2c2064f24a70ef3 P 1a2ecd3c1ea24075a665a1b393160c8d: No bootstrap required, opened a new log
I20260810 07:18:46.793838 5329 ts_tablet_manager.cc:1397] T b33807f9eb084b12b2c2064f24a70ef3 P 1a2ecd3c1ea24075a665a1b393160c8d: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260810 07:18:46.793852 5328 raft_consensus.cc:348] T b33807f9eb084b12b2c2064f24a70ef3 P f46685e09a094f8195ad61413d461d6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46685e09a094f8195ad61413d461d6c" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33963 } } peers { permanent_uuid: "aad136faefcb429596c6e295a69eb365" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 38833 } } peers { permanent_uuid: "1a2ecd3c1ea24075a665a1b393160c8d" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 40181 } }
I20260810 07:18:46.794077 5328 raft_consensus.cc:374] T b33807f9eb084b12b2c2064f24a70ef3 P f46685e09a094f8195ad61413d461d6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:46.794145 5328 raft_consensus.cc:729] T b33807f9eb084b12b2c2064f24a70ef3 P f46685e09a094f8195ad61413d461d6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f46685e09a094f8195ad61413d461d6c, State: Initialized, Role: FOLLOWER
I20260810 07:18:46.794126 5327 raft_consensus.cc:348] T b33807f9eb084b12b2c2064f24a70ef3 P aad136faefcb429596c6e295a69eb365 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46685e09a094f8195ad61413d461d6c" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33963 } } peers { permanent_uuid: "aad136faefcb429596c6e295a69eb365" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 38833 } } peers { permanent_uuid: "1a2ecd3c1ea24075a665a1b393160c8d" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 40181 } }
I20260810 07:18:46.794310 5327 raft_consensus.cc:374] T b33807f9eb084b12b2c2064f24a70ef3 P aad136faefcb429596c6e295a69eb365 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:46.794282 5328 consensus_queue.cc:260] T b33807f9eb084b12b2c2064f24a70ef3 P f46685e09a094f8195ad61413d461d6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46685e09a094f8195ad61413d461d6c" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33963 } } peers { permanent_uuid: "aad136faefcb429596c6e295a69eb365" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 38833 } } peers { permanent_uuid: "1a2ecd3c1ea24075a665a1b393160c8d" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 40181 } }
I20260810 07:18:46.794407 5327 raft_consensus.cc:729] T b33807f9eb084b12b2c2064f24a70ef3 P aad136faefcb429596c6e295a69eb365 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aad136faefcb429596c6e295a69eb365, State: Initialized, Role: FOLLOWER
I20260810 07:18:46.794533 5327 consensus_queue.cc:260] T b33807f9eb084b12b2c2064f24a70ef3 P aad136faefcb429596c6e295a69eb365 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46685e09a094f8195ad61413d461d6c" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33963 } } peers { permanent_uuid: "aad136faefcb429596c6e295a69eb365" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 38833 } } peers { permanent_uuid: "1a2ecd3c1ea24075a665a1b393160c8d" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 40181 } }
I20260810 07:18:46.794592 5328 ts_tablet_manager.cc:1428] T b33807f9eb084b12b2c2064f24a70ef3 P f46685e09a094f8195ad61413d461d6c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:18:46.794740 5161 heartbeater.cc:503] Master 127.15.217.126:46095 was elected leader, sending a full tablet report...
I20260810 07:18:46.794813 5327 ts_tablet_manager.cc:1428] T b33807f9eb084b12b2c2064f24a70ef3 P aad136faefcb429596c6e295a69eb365: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:18:46.794837 5309 heartbeater.cc:503] Master 127.15.217.126:46095 was elected leader, sending a full tablet report...
I20260810 07:18:46.796079 5329 raft_consensus.cc:348] T b33807f9eb084b12b2c2064f24a70ef3 P 1a2ecd3c1ea24075a665a1b393160c8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46685e09a094f8195ad61413d461d6c" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33963 } } peers { permanent_uuid: "aad136faefcb429596c6e295a69eb365" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 38833 } } peers { permanent_uuid: "1a2ecd3c1ea24075a665a1b393160c8d" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 40181 } }
I20260810 07:18:46.796262 5329 raft_consensus.cc:374] T b33807f9eb084b12b2c2064f24a70ef3 P 1a2ecd3c1ea24075a665a1b393160c8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:46.796311 5329 raft_consensus.cc:729] T b33807f9eb084b12b2c2064f24a70ef3 P 1a2ecd3c1ea24075a665a1b393160c8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a2ecd3c1ea24075a665a1b393160c8d, State: Initialized, Role: FOLLOWER
I20260810 07:18:46.796427 5329 consensus_queue.cc:260] T b33807f9eb084b12b2c2064f24a70ef3 P 1a2ecd3c1ea24075a665a1b393160c8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46685e09a094f8195ad61413d461d6c" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33963 } } peers { permanent_uuid: "aad136faefcb429596c6e295a69eb365" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 38833 } } peers { permanent_uuid: "1a2ecd3c1ea24075a665a1b393160c8d" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 40181 } }
I20260810 07:18:46.796747 5329 ts_tablet_manager.cc:1428] T b33807f9eb084b12b2c2064f24a70ef3 P 1a2ecd3c1ea24075a665a1b393160c8d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:18:46.796775 5017 heartbeater.cc:503] Master 127.15.217.126:46095 was elected leader, sending a full tablet report...
W20260810 07:18:46.827337 5162 tablet.cc:2369] T b33807f9eb084b12b2c2064f24a70ef3 P f46685e09a094f8195ad61413d461d6c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:18:46.884366 5018 tablet.cc:2369] T b33807f9eb084b12b2c2064f24a70ef3 P 1a2ecd3c1ea24075a665a1b393160c8d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:18:46.998340 5337 raft_consensus.cc:482] T b33807f9eb084b12b2c2064f24a70ef3 P 1a2ecd3c1ea24075a665a1b393160c8d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:18:46.998543 5337 raft_consensus.cc:504] T b33807f9eb084b12b2c2064f24a70ef3 P 1a2ecd3c1ea24075a665a1b393160c8d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46685e09a094f8195ad61413d461d6c" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33963 } } peers { permanent_uuid: "aad136faefcb429596c6e295a69eb365" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 38833 } } peers { permanent_uuid: "1a2ecd3c1ea24075a665a1b393160c8d" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 40181 } }
W20260810 07:18:46.998792 5310 tablet.cc:2369] T b33807f9eb084b12b2c2064f24a70ef3 P aad136faefcb429596c6e295a69eb365: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:18:46.999083 5337 leader_election.cc:288] T b33807f9eb084b12b2c2064f24a70ef3 P 1a2ecd3c1ea24075a665a1b393160c8d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f46685e09a094f8195ad61413d461d6c (127.15.217.66:33963), aad136faefcb429596c6e295a69eb365 (127.15.217.67:38833)
I20260810 07:18:47.002203 5323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:46.999384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:41115 (local address 127.15.217.67:38833)
0810 07:18:46.999439 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:46.999441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:46.999529 (+ 88us) server_negotiation.cc:407] Connection header received
0810 07:18:46.999616 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:46.999617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:46.999627 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:46.999690 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:46.999874 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.000359 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.001412 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.001833 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.001924 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:47.001935 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:47.002093 (+ 158us) server_negotiation.cc:299] Negotiation successful
0810 07:18:47.002133 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 07:18:47.002231 5339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:46.999297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:38833 (local address 127.15.217.65:41115)
0810 07:18:46.999444 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:46.999454 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:46.999548 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:46.999701 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:46.999702 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:46.999703 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:46.999794 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:46.999796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.000481 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.000483 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:47.001300 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:47.001303 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.001924 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.001925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:47.002015 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:47.002019 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:18:47.002087 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 07:18:47.002130 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":81,"threads_started":1}
I20260810 07:18:47.002543 5338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:46.999258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:33963 (local address 127.15.217.65:55367)
0810 07:18:46.999389 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:46.999402 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:46.999482 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:46.999615 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:46.999616 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:46.999618 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:46.999702 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:46.999705 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.000623 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.000624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:47.001386 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:47.001387 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.002216 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.002217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:47.002310 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:47.002316 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 07:18:47.002392 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 07:18:47.002455 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":61,"threads_started":1}
I20260810 07:18:47.002543 5325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:46.999366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:55367 (local address 127.15.217.66:33963)
0810 07:18:46.999409 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:46.999411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:46.999463 (+ 52us) server_negotiation.cc:407] Connection header received
0810 07:18:46.999541 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:46.999542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:46.999554 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:46.999602 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:46.999784 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.000509 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.001635 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.002114 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.002215 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:47.002225 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:47.002414 (+ 189us) server_negotiation.cc:299] Negotiation successful
0810 07:18:47.002471 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 07:18:47.002660 5260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b33807f9eb084b12b2c2064f24a70ef3" candidate_uuid: "1a2ecd3c1ea24075a665a1b393160c8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aad136faefcb429596c6e295a69eb365" is_pre_election: true
I20260810 07:18:47.002866 5260 raft_consensus.cc:2454] T b33807f9eb084b12b2c2064f24a70ef3 P aad136faefcb429596c6e295a69eb365 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a2ecd3c1ea24075a665a1b393160c8d in term 0.
I20260810 07:18:47.003113 4897 leader_election.cc:302] T b33807f9eb084b12b2c2064f24a70ef3 P 1a2ecd3c1ea24075a665a1b393160c8d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a2ecd3c1ea24075a665a1b393160c8d, aad136faefcb429596c6e295a69eb365; no voters:
I20260810 07:18:47.003161 5113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b33807f9eb084b12b2c2064f24a70ef3" candidate_uuid: "1a2ecd3c1ea24075a665a1b393160c8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46685e09a094f8195ad61413d461d6c" is_pre_election: true
I20260810 07:18:47.003345 5113 raft_consensus.cc:2454] T b33807f9eb084b12b2c2064f24a70ef3 P f46685e09a094f8195ad61413d461d6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a2ecd3c1ea24075a665a1b393160c8d in term 0.
I20260810 07:18:47.003355 5337 raft_consensus.cc:2781] T b33807f9eb084b12b2c2064f24a70ef3 P 1a2ecd3c1ea24075a665a1b393160c8d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:18:47.003427 5337 raft_consensus.cc:482] T b33807f9eb084b12b2c2064f24a70ef3 P 1a2ecd3c1ea24075a665a1b393160c8d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:18:47.003463 5337 raft_consensus.cc:3037] T b33807f9eb084b12b2c2064f24a70ef3 P 1a2ecd3c1ea24075a665a1b393160c8d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:47.004408 5337 raft_consensus.cc:504] T b33807f9eb084b12b2c2064f24a70ef3 P 1a2ecd3c1ea24075a665a1b393160c8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46685e09a094f8195ad61413d461d6c" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33963 } } peers { permanent_uuid: "aad136faefcb429596c6e295a69eb365" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 38833 } } peers { permanent_uuid: "1a2ecd3c1ea24075a665a1b393160c8d" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 40181 } }
I20260810 07:18:47.004648 5337 leader_election.cc:288] T b33807f9eb084b12b2c2064f24a70ef3 P 1a2ecd3c1ea24075a665a1b393160c8d [CANDIDATE]: Term 1 election: Requested vote from peers f46685e09a094f8195ad61413d461d6c (127.15.217.66:33963), aad136faefcb429596c6e295a69eb365 (127.15.217.67:38833)
I20260810 07:18:47.004825 5113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b33807f9eb084b12b2c2064f24a70ef3" candidate_uuid: "1a2ecd3c1ea24075a665a1b393160c8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46685e09a094f8195ad61413d461d6c"
I20260810 07:18:47.004834 5260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b33807f9eb084b12b2c2064f24a70ef3" candidate_uuid: "1a2ecd3c1ea24075a665a1b393160c8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aad136faefcb429596c6e295a69eb365"
I20260810 07:18:47.004921 5260 raft_consensus.cc:3037] T b33807f9eb084b12b2c2064f24a70ef3 P aad136faefcb429596c6e295a69eb365 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:47.004921 5113 raft_consensus.cc:3037] T b33807f9eb084b12b2c2064f24a70ef3 P f46685e09a094f8195ad61413d461d6c [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:47.005996 5260 raft_consensus.cc:2454] T b33807f9eb084b12b2c2064f24a70ef3 P aad136faefcb429596c6e295a69eb365 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a2ecd3c1ea24075a665a1b393160c8d in term 1.
I20260810 07:18:47.005995 5113 raft_consensus.cc:2454] T b33807f9eb084b12b2c2064f24a70ef3 P f46685e09a094f8195ad61413d461d6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a2ecd3c1ea24075a665a1b393160c8d in term 1.
I20260810 07:18:47.006214 4895 leader_election.cc:302] T b33807f9eb084b12b2c2064f24a70ef3 P 1a2ecd3c1ea24075a665a1b393160c8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a2ecd3c1ea24075a665a1b393160c8d, f46685e09a094f8195ad61413d461d6c; no voters:
I20260810 07:18:47.006316 5337 raft_consensus.cc:2781] T b33807f9eb084b12b2c2064f24a70ef3 P 1a2ecd3c1ea24075a665a1b393160c8d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:18:47.006502 5337 raft_consensus.cc:686] T b33807f9eb084b12b2c2064f24a70ef3 P 1a2ecd3c1ea24075a665a1b393160c8d [term 1 LEADER]: Becoming Leader. State: Replica: 1a2ecd3c1ea24075a665a1b393160c8d, State: Running, Role: LEADER
I20260810 07:18:47.006760 5337 consensus_queue.cc:237] T b33807f9eb084b12b2c2064f24a70ef3 P 1a2ecd3c1ea24075a665a1b393160c8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46685e09a094f8195ad61413d461d6c" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33963 } } peers { permanent_uuid: "aad136faefcb429596c6e295a69eb365" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 38833 } } peers { permanent_uuid: "1a2ecd3c1ea24075a665a1b393160c8d" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 40181 } }
I20260810 07:18:47.007829 4811 catalog_manager.cc:5495] T b33807f9eb084b12b2c2064f24a70ef3 P 1a2ecd3c1ea24075a665a1b393160c8d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a2ecd3c1ea24075a665a1b393160c8d (127.15.217.65). New cstate: current_term: 1 leader_uuid: "1a2ecd3c1ea24075a665a1b393160c8d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46685e09a094f8195ad61413d461d6c" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aad136faefcb429596c6e295a69eb365" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 38833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a2ecd3c1ea24075a665a1b393160c8d" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 40181 } health_report { overall_health: HEALTHY } } }
I20260810 07:18:47.068482 5326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:47.066102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51110 (local address 127.15.217.65:40181)
0810 07:18:47.066177 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:47.066180 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:47.066191 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:18:47.066282 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:47.066283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:47.066306 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:47.066362 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:18:47.066524 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.066988 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.067728 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.067917 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.067969 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:47.068121 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:18:47.068235 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:18:47.068299 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:47.068335 (+ 36us) server_negotiation.cc:299] Negotiation successful
0810 07:18:47.068352 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:18:47.070598 5113 raft_consensus.cc:1261] T b33807f9eb084b12b2c2064f24a70ef3 P f46685e09a094f8195ad61413d461d6c [term 1 FOLLOWER]: Refusing update from remote peer 1a2ecd3c1ea24075a665a1b393160c8d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:18:47.070600 5260 raft_consensus.cc:1261] T b33807f9eb084b12b2c2064f24a70ef3 P aad136faefcb429596c6e295a69eb365 [term 1 FOLLOWER]: Refusing update from remote peer 1a2ecd3c1ea24075a665a1b393160c8d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:18:47.071020 5341 consensus_queue.cc:1035] T b33807f9eb084b12b2c2064f24a70ef3 P 1a2ecd3c1ea24075a665a1b393160c8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "aad136faefcb429596c6e295a69eb365" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 38833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:18:47.071180 5337 consensus_queue.cc:1035] T b33807f9eb084b12b2c2064f24a70ef3 P 1a2ecd3c1ea24075a665a1b393160c8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f46685e09a094f8195ad61413d461d6c" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 33963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:18:47.074028 5346 mvcc.cc:205] Tried to move back new op lower bound from 7316874555677872128 to 7316874555420463104. Current Snapshot: MvccSnapshot[applied={T|T < 7316874555677872128}]
I20260810 07:18:47.074955 5347 mvcc.cc:205] Tried to move back new op lower bound from 7316874555677872128 to 7316874555420463104. Current Snapshot: MvccSnapshot[applied={T|T < 7316874555677872128 or (T in {7316874555677872128})}]
I20260810 07:18:47.076030 4810 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53992:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:18:47.076146 4810 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53992:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:18:47.076943 4810 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4c3e5c8a2cf44a198738bb7325152992: Sending DeleteTablet for 3 replicas of tablet b33807f9eb084b12b2c2064f24a70ef3
I20260810 07:18:47.077350 5091 tablet_service.cc:1514] Processing DeleteTablet for tablet b33807f9eb084b12b2c2064f24a70ef3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:18:47 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:35642
I20260810 07:18:47.077354 5239 tablet_service.cc:1514] Processing DeleteTablet for tablet b33807f9eb084b12b2c2064f24a70ef3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:18:47 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:44020
I20260810 07:18:47.077389 4947 tablet_service.cc:1514] Processing DeleteTablet for tablet b33807f9eb084b12b2c2064f24a70ef3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:18:47 UTC) from {username='kudu', principal='kudu/127.15.217.126@KRBTEST.COM'} at 127.0.0.1:51108
I20260810 07:18:47.077605 5355 tablet_replica.cc:330] T b33807f9eb084b12b2c2064f24a70ef3 P 1a2ecd3c1ea24075a665a1b393160c8d: stopping tablet replica
I20260810 07:18:47.077749 5355 raft_consensus.cc:2229] T b33807f9eb084b12b2c2064f24a70ef3 P 1a2ecd3c1ea24075a665a1b393160c8d [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:18:47.077903 5355 raft_consensus.cc:2258] T b33807f9eb084b12b2c2064f24a70ef3 P 1a2ecd3c1ea24075a665a1b393160c8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:18:47.077956 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 4878
I20260810 07:18:47.084714 5356 tablet_replica.cc:330] T b33807f9eb084b12b2c2064f24a70ef3 P aad136faefcb429596c6e295a69eb365: stopping tablet replica
I20260810 07:18:47.084717 5354 tablet_replica.cc:330] T b33807f9eb084b12b2c2064f24a70ef3 P f46685e09a094f8195ad61413d461d6c: stopping tablet replica
I20260810 07:18:47.084903 5356 raft_consensus.cc:2229] T b33807f9eb084b12b2c2064f24a70ef3 P aad136faefcb429596c6e295a69eb365 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:18:47.084909 5354 raft_consensus.cc:2229] T b33807f9eb084b12b2c2064f24a70ef3 P f46685e09a094f8195ad61413d461d6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:18:47.085069 5354 raft_consensus.cc:2258] T b33807f9eb084b12b2c2064f24a70ef3 P f46685e09a094f8195ad61413d461d6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:18:47.085103 5356 raft_consensus.cc:2258] T b33807f9eb084b12b2c2064f24a70ef3 P aad136faefcb429596c6e295a69eb365 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:18:47.085636 5356 ts_tablet_manager.cc:1905] T b33807f9eb084b12b2c2064f24a70ef3 P aad136faefcb429596c6e295a69eb365: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:18:47.085641 5354 ts_tablet_manager.cc:1905] T b33807f9eb084b12b2c2064f24a70ef3 P f46685e09a094f8195ad61413d461d6c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:18:47.086931 4794 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.217.65:40181 (error 108)
I20260810 07:18:47.087209 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 5023
I20260810 07:18:47.088477 5356 ts_tablet_manager.cc:1918] T b33807f9eb084b12b2c2064f24a70ef3 P aad136faefcb429596c6e295a69eb365: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:18:47.088606 5356 log.cc:1196] T b33807f9eb084b12b2c2064f24a70ef3 P aad136faefcb429596c6e295a69eb365: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346081759552-16229-0/minicluster-data/ts-2/wal/wals/b33807f9eb084b12b2c2064f24a70ef3
I20260810 07:18:47.088972 5356 ts_tablet_manager.cc:1939] T b33807f9eb084b12b2c2064f24a70ef3 P aad136faefcb429596c6e295a69eb365: Deleting consensus metadata
I20260810 07:18:47.090279 4796 catalog_manager.cc:4841] TS aad136faefcb429596c6e295a69eb365 (127.15.217.67:38833): tablet b33807f9eb084b12b2c2064f24a70ef3 (table test-table [id=a084a9abda38416d84108fd131df879e]) successfully deleted
I20260810 07:18:47.091923 5321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:47.091765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:40181 (local address 127.0.0.1:51118)
0810 07:18:47.091829 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:47.091879 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.65:40181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 07:18:47.092195 4794 catalog_manager.cc:4584] TS 1a2ecd3c1ea24075a665a1b393160c8d (127.15.217.65:40181): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b33807f9eb084b12b2c2064f24a70ef3: Network error: Client connection negotiation failed: client connection to 127.15.217.65:40181: connect: Connection refused (error 111)
I20260810 07:18:47.096849 5321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:47.096688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:33963 (local address 127.0.0.1:35644)
0810 07:18:47.096767 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:47.096807 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.66:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260810 07:18:47.096829 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 5166
I20260810 07:18:47.102950 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 4776
2026-08-10T07:18:47Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2348 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-10T07:18:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:18:47Z Disabled control of system clock
I20260810 07:18:47.135906 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:39485
--webserver_interface=127.15.217.126
--webserver_port=0
--builtin_ntp_servers=127.15.217.84:39383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:39485
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 07:18:47.222229 5361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:47.222488 5361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:47.222512 5361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:47.223645 5361 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:18:47.223682 5361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:47.223695 5361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:47.223708 5361 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:18:47.223721 5361 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:18:47.225060 5361 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:39383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:39485
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:39485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:47.225274 5361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:47.225536 5361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:18:47.230715 5361 server_base.cc:1034] running on GCE node
W20260810 07:18:47.230708 5370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:47.230657 5366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:47.230664 5367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:47.231264 5361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:47.231639 5361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:47.232817 5361 hybrid_clock.cc:648] HybridClock initialized: now 1786346327232811 us; error 39 us; skew 500 ppm
I20260810 07:18:47.234135 5361 webserver.cc:460] Webserver started at http://127.15.217.126:42347/ using document root <none> and password file <none>
I20260810 07:18:47.234405 5361 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:47.234479 5361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:47.234642 5361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:47.235806 5361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "d0b4f73098624b629c54e18095eb2ae5"
format_stamp: "Formatted at 2026-08-10 07:18:47 on dist-test-slave-10pc"
server_key: "1129ac9ad00c7ce261be28f52c4415be"
server_key_iv: "3e4493091d3fe434dab587637da4ff04"
server_key_version: "encryptionkey@0"
I20260810 07:18:47.236212 5361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "d0b4f73098624b629c54e18095eb2ae5"
format_stamp: "Formatted at 2026-08-10 07:18:47 on dist-test-slave-10pc"
server_key: "1129ac9ad00c7ce261be28f52c4415be"
server_key_iv: "3e4493091d3fe434dab587637da4ff04"
server_key_version: "encryptionkey@0"
I20260810 07:18:47.237788 5361 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:18:47.238644 5376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:47.238859 5361 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:18:47.238972 5361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "d0b4f73098624b629c54e18095eb2ae5"
format_stamp: "Formatted at 2026-08-10 07:18:47 on dist-test-slave-10pc"
server_key: "1129ac9ad00c7ce261be28f52c4415be"
server_key_iv: "3e4493091d3fe434dab587637da4ff04"
server_key_version: "encryptionkey@0"
I20260810 07:18:47.239054 5361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:47.257519 5361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:47.257908 5361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:47.258095 5361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:47.262317 5361 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:39485
I20260810 07:18:47.262362 5432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:39485 every 8 connection(s)
I20260810 07:18:47.262771 5361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:18:47.263922 5433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:47.266777 5433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0b4f73098624b629c54e18095eb2ae5: Bootstrap starting.
I20260810 07:18:47.267508 5433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0b4f73098624b629c54e18095eb2ae5: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:47.267860 5433 log.cc:824] T 00000000000000000000000000000000 P d0b4f73098624b629c54e18095eb2ae5: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:47.268774 5433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0b4f73098624b629c54e18095eb2ae5: No bootstrap required, opened a new log
I20260810 07:18:47.270659 5433 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0b4f73098624b629c54e18095eb2ae5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0b4f73098624b629c54e18095eb2ae5" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 39485 } }
I20260810 07:18:47.270833 5433 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0b4f73098624b629c54e18095eb2ae5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:47.270897 5433 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0b4f73098624b629c54e18095eb2ae5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0b4f73098624b629c54e18095eb2ae5, State: Initialized, Role: FOLLOWER
I20260810 07:18:47.271088 5433 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0b4f73098624b629c54e18095eb2ae5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0b4f73098624b629c54e18095eb2ae5" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 39485 } }
I20260810 07:18:47.271189 5433 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0b4f73098624b629c54e18095eb2ae5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:18:47.271219 5433 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0b4f73098624b629c54e18095eb2ae5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:18:47.271284 5433 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0b4f73098624b629c54e18095eb2ae5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:47.271284 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 5361
I20260810 07:18:47.271412 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:18:47.271628 16229 external_mini_cluster.cc:1428] Setting key 3b0386b0fa2656c84b9402df066e3f94
I20260810 07:18:47.272096 5433 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0b4f73098624b629c54e18095eb2ae5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0b4f73098624b629c54e18095eb2ae5" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 39485 } }
I20260810 07:18:47.272289 5433 leader_election.cc:302] T 00000000000000000000000000000000 P d0b4f73098624b629c54e18095eb2ae5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0b4f73098624b629c54e18095eb2ae5; no voters:
I20260810 07:18:47.272545 5433 leader_election.cc:288] T 00000000000000000000000000000000 P d0b4f73098624b629c54e18095eb2ae5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:18:47.272562 5438 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0b4f73098624b629c54e18095eb2ae5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:18:47.272770 5438 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0b4f73098624b629c54e18095eb2ae5 [term 1 LEADER]: Becoming Leader. State: Replica: d0b4f73098624b629c54e18095eb2ae5, State: Running, Role: LEADER
I20260810 07:18:47.272882 5433 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0b4f73098624b629c54e18095eb2ae5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:18:47.272933 5438 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0b4f73098624b629c54e18095eb2ae5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0b4f73098624b629c54e18095eb2ae5" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 39485 } }
I20260810 07:18:47.273540 5440 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0b4f73098624b629c54e18095eb2ae5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0b4f73098624b629c54e18095eb2ae5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0b4f73098624b629c54e18095eb2ae5" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 39485 } } }
I20260810 07:18:47.273646 5440 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0b4f73098624b629c54e18095eb2ae5 [sys.catalog]: This master's current role is: LEADER
I20260810 07:18:47.274124 5450 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:18:47.274140 5441 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0b4f73098624b629c54e18095eb2ae5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0b4f73098624b629c54e18095eb2ae5. Latest consensus state: current_term: 1 leader_uuid: "d0b4f73098624b629c54e18095eb2ae5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0b4f73098624b629c54e18095eb2ae5" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 39485 } } }
I20260810 07:18:47.274348 5441 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0b4f73098624b629c54e18095eb2ae5 [sys.catalog]: This master's current role is: LEADER
I20260810 07:18:47.274701 5450 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:18:47.276759 5450 catalog_manager.cc:1347] Generated new cluster ID: 460f177ddefb41a09c888fb4290b388f
I20260810 07:18:47.276834 5450 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:18:47.278820 5437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:47.272243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34110 (local address 127.15.217.126:39485)
0810 07:18:47.272571 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:47.272583 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:47.272625 (+ 42us) server_negotiation.cc:407] Connection header received
0810 07:18:47.272793 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:47.272802 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:47.272895 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:47.273393 (+ 498us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:47.273511 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.275263 (+ 1752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.276391 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.277052 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.277185 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:47.278009 (+ 824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:47.278013 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:47.278014 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:18:47.278240 (+ 226us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:47.278283 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:18:47.278298 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:47.278371 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:47.278465 (+ 94us) server_negotiation.cc:299] Negotiation successful
0810 07:18:47.278559 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":157,"threads_started":1}
I20260810 07:18:47.294430 5450 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:18:47.295292 5450 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:18:47.303558 5450 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0b4f73098624b629c54e18095eb2ae5: Generated new TSK 0
I20260810 07:18:47.303810 5450 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:18:47.305660 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.65:0
--local_ip_for_outbound_sockets=127.15.217.65
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:39485
--builtin_ntp_servers=127.15.217.84:39383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 07:18:47.402731 5461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:47.403004 5461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:47.403045 5461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:47.404325 5461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:47.404397 5461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:47.404443 5461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.65
I20260810 07:18:47.405931 5461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:39383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.217.65
--webserver_port=0
--tserver_master_addrs=127.15.217.126:39485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.65
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:47.406230 5461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:47.406564 5461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:47.412820 5467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:47.412811 5469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:47.412813 5466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:47.413092 5461 server_base.cc:1034] running on GCE node
I20260810 07:18:47.413384 5461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:47.413695 5461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:47.414889 5461 hybrid_clock.cc:648] HybridClock initialized: now 1786346327414860 us; error 44 us; skew 500 ppm
I20260810 07:18:47.416534 5461 webserver.cc:460] Webserver started at http://127.15.217.65:37915/ using document root <none> and password file <none>
I20260810 07:18:47.416854 5461 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:47.416921 5461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:47.417047 5461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:47.418560 5461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-0/data/instance:
uuid: "e06e739277014c43b9c77401cfe135bb"
format_stamp: "Formatted at 2026-08-10 07:18:47 on dist-test-slave-10pc"
server_key: "e6460ff8ea6b9608cca2ed30ce450906"
server_key_iv: "dbec35b48ea314e831f30eca3265f847"
server_key_version: "encryptionkey@0"
I20260810 07:18:47.419019 5461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance:
uuid: "e06e739277014c43b9c77401cfe135bb"
format_stamp: "Formatted at 2026-08-10 07:18:47 on dist-test-slave-10pc"
server_key: "e6460ff8ea6b9608cca2ed30ce450906"
server_key_iv: "dbec35b48ea314e831f30eca3265f847"
server_key_version: "encryptionkey@0"
I20260810 07:18:47.420418 5461 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260810 07:18:47.421404 5476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:47.421883 5461 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:18:47.421984 5461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-0/wal
uuid: "e06e739277014c43b9c77401cfe135bb"
format_stamp: "Formatted at 2026-08-10 07:18:47 on dist-test-slave-10pc"
server_key: "e6460ff8ea6b9608cca2ed30ce450906"
server_key_iv: "dbec35b48ea314e831f30eca3265f847"
server_key_version: "encryptionkey@0"
I20260810 07:18:47.422106 5461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:47.436374 5461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:47.436841 5461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:47.437012 5461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:47.437282 5461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:18:47.437651 5461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:18:47.437686 5461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:47.437711 5461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:18:47.437758 5461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:47.444500 5461 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.65:34141
I20260810 07:18:47.444535 5600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.65:34141 every 8 connection(s)
I20260810 07:18:47.444952 5461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-0/data/info.pb
I20260810 07:18:47.450486 5437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:47.445807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.65:52945 (local address 127.15.217.126:39485)
0810 07:18:47.445884 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:47.445886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:47.446156 (+ 270us) server_negotiation.cc:407] Connection header received
0810 07:18:47.446669 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:47.446670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:47.446683 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:47.446751 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:47.447457 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.447971 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.449117 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.449316 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.449374 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:47.450227 (+ 853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:47.450229 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:47.450229 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:18:47.450235 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:47.450250 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:18:47.450254 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:47.450299 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:47.450389 (+ 90us) server_negotiation.cc:299] Negotiation successful
0810 07:18:47.450421 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:18:47.450728 5604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:47.445786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:39485 (local address 127.15.217.65:52945)
0810 07:18:47.446008 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:47.446052 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:47.446577 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:47.446808 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:47.446811 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:47.446819 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:18:47.447372 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:47.447377 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.448079 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.448080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:47.449002 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:47.449005 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.449399 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.449399 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:47.449518 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:47.450053 (+ 535us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:18:47.450053 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:18:47.450083 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:18:47.450135 (+ 52us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:18:47.450312 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:18:47.450317 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:18:47.450320 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:18:47.450373 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 07:18:47.450462 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":117,"threads_started":1}
I20260810 07:18:47.451429 5601 heartbeater.cc:347] Connected to a master server at 127.15.217.126:39485
I20260810 07:18:47.451526 5601 heartbeater.cc:464] Registering TS with master...
I20260810 07:18:47.451756 5601 heartbeater.cc:511] Master 127.15.217.126:39485 requested a full tablet report, sending...
I20260810 07:18:47.452438 5393 ts_manager.cc:194] Registered new tserver with Master: e06e739277014c43b9c77401cfe135bb (127.15.217.65:34141)
I20260810 07:18:47.453128 5393 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.217.65:52945
I20260810 07:18:47.453446 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 5461
I20260810 07:18:47.453557 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-0/wal/instance
I20260810 07:18:47.453787 16229 external_mini_cluster.cc:1428] Setting key cc6c25d2c041bc22e688c71ae46f232c
I20260810 07:18:47.454900 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.66:0
--local_ip_for_outbound_sockets=127.15.217.66
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:39485
--builtin_ntp_servers=127.15.217.84:39383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 07:18:47.545454 5605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:47.545689 5605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:47.545711 5605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:47.546880 5605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:47.546917 5605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:47.546952 5605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.66
I20260810 07:18:47.548309 5605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:39383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.217.66
--webserver_port=0
--tserver_master_addrs=127.15.217.126:39485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.66
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:47.548616 5605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:47.548921 5605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:47.554739 5610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:47.554764 5611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:47.554739 5614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:47.555058 5605 server_base.cc:1034] running on GCE node
I20260810 07:18:47.555388 5605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:47.555624 5605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:47.556804 5605 hybrid_clock.cc:648] HybridClock initialized: now 1786346327556792 us; error 32 us; skew 500 ppm
I20260810 07:18:47.558564 5605 webserver.cc:460] Webserver started at http://127.15.217.66:34089/ using document root <none> and password file <none>
I20260810 07:18:47.558856 5605 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:47.558907 5605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:47.559134 5605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:47.560312 5605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-1/data/instance:
uuid: "93e547007e2141908f72433c7ababbf1"
format_stamp: "Formatted at 2026-08-10 07:18:47 on dist-test-slave-10pc"
server_key: "69640ccdb080e842dadc0e8b4fdec86e"
server_key_iv: "73b4167865cb963fa4533496abf7a51f"
server_key_version: "encryptionkey@0"
I20260810 07:18:47.560755 5605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance:
uuid: "93e547007e2141908f72433c7ababbf1"
format_stamp: "Formatted at 2026-08-10 07:18:47 on dist-test-slave-10pc"
server_key: "69640ccdb080e842dadc0e8b4fdec86e"
server_key_iv: "73b4167865cb963fa4533496abf7a51f"
server_key_version: "encryptionkey@0"
I20260810 07:18:47.562156 5605 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:18:47.562958 5620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:47.563144 5605 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 07:18:47.563264 5605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-1/wal
uuid: "93e547007e2141908f72433c7ababbf1"
format_stamp: "Formatted at 2026-08-10 07:18:47 on dist-test-slave-10pc"
server_key: "69640ccdb080e842dadc0e8b4fdec86e"
server_key_iv: "73b4167865cb963fa4533496abf7a51f"
server_key_version: "encryptionkey@0"
I20260810 07:18:47.563345 5605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:47.581005 5605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:47.581398 5605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:47.581585 5605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:47.581853 5605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:18:47.582250 5605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:18:47.582311 5605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:47.582347 5605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:18:47.582376 5605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:47.588502 5605 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.66:40361
I20260810 07:18:47.588531 5741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.66:40361 every 8 connection(s)
I20260810 07:18:47.588927 5605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-1/data/info.pb
I20260810 07:18:47.591683 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 5605
I20260810 07:18:47.591820 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-1/wal/instance
I20260810 07:18:47.592039 16229 external_mini_cluster.cc:1428] Setting key 434e26e79aaac268f0f624a165f4e244
I20260810 07:18:47.593139 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.217.67:0
--local_ip_for_outbound_sockets=127.15.217.67
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:39485
--builtin_ntp_servers=127.15.217.84:39383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 07:18:47.595108 5437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:47.589727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:45033 (local address 127.15.217.126:39485)
0810 07:18:47.589820 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:47.589823 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:47.590037 (+ 214us) server_negotiation.cc:407] Connection header received
0810 07:18:47.590547 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:47.590549 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:47.590559 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:47.590615 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:47.591383 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.591959 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.593518 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.593731 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.593801 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:47.594795 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:47.594797 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:47.594797 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:18:47.594802 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:47.594818 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:18:47.594822 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:47.594871 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:47.595009 (+ 138us) server_negotiation.cc:299] Negotiation successful
0810 07:18:47.595044 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 07:18:47.595383 5744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:47.589682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:39485 (local address 127.15.217.66:45033)
0810 07:18:47.589899 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:47.589937 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:47.590453 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:47.590717 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:47.590721 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:47.590730 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:18:47.591284 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:47.591288 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.592067 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.592069 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:47.593404 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:47.593408 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.593828 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.593830 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:47.594001 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:47.594613 (+ 612us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:18:47.594613 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:18:47.594656 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:18:47.594697 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:18:47.594896 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:18:47.594904 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:18:47.594908 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:18:47.594991 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 07:18:47.595125 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":104,"threads_started":1}
I20260810 07:18:47.595932 5742 heartbeater.cc:347] Connected to a master server at 127.15.217.126:39485
I20260810 07:18:47.596092 5742 heartbeater.cc:464] Registering TS with master...
I20260810 07:18:47.596328 5742 heartbeater.cc:511] Master 127.15.217.126:39485 requested a full tablet report, sending...
I20260810 07:18:47.596823 5393 ts_manager.cc:194] Registered new tserver with Master: 93e547007e2141908f72433c7ababbf1 (127.15.217.66:40361)
I20260810 07:18:47.597235 5393 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.217.66:45033
W20260810 07:18:47.683080 5745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:47.683357 5745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:47.683394 5745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:47.684603 5745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:47.684675 5745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:47.684719 5745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.217.67
I20260810 07:18:47.686159 5745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:39383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.217.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.217.67
--webserver_port=0
--tserver_master_addrs=127.15.217.126:39485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.217.67
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:47.686474 5745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:47.686839 5745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:47.692848 5754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:47.692898 5752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:47.692847 5751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:47.693132 5745 server_base.cc:1034] running on GCE node
I20260810 07:18:47.693439 5745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:47.693739 5745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:47.694963 5745 hybrid_clock.cc:648] HybridClock initialized: now 1786346327694950 us; error 51 us; skew 500 ppm
I20260810 07:18:47.696514 5745 webserver.cc:460] Webserver started at http://127.15.217.67:40247/ using document root <none> and password file <none>
I20260810 07:18:47.696810 5745 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:47.696884 5745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:47.697031 5745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:47.698210 5745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-2/data/instance:
uuid: "5db098d08e0b4c3d835c9bb46bb342b5"
format_stamp: "Formatted at 2026-08-10 07:18:47 on dist-test-slave-10pc"
server_key: "ed523951b59c4833a02ce46c03a4b252"
server_key_iv: "24cee6a87c8ad937fff190251f3a145f"
server_key_version: "encryptionkey@0"
I20260810 07:18:47.698637 5745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance:
uuid: "5db098d08e0b4c3d835c9bb46bb342b5"
format_stamp: "Formatted at 2026-08-10 07:18:47 on dist-test-slave-10pc"
server_key: "ed523951b59c4833a02ce46c03a4b252"
server_key_iv: "24cee6a87c8ad937fff190251f3a145f"
server_key_version: "encryptionkey@0"
I20260810 07:18:47.700053 5745 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:18:47.700909 5760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:47.701094 5745 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:47.701198 5745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-2/wal
uuid: "5db098d08e0b4c3d835c9bb46bb342b5"
format_stamp: "Formatted at 2026-08-10 07:18:47 on dist-test-slave-10pc"
server_key: "ed523951b59c4833a02ce46c03a4b252"
server_key_iv: "24cee6a87c8ad937fff190251f3a145f"
server_key_version: "encryptionkey@0"
I20260810 07:18:47.701287 5745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:47.712261 5745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:47.712711 5745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:47.712908 5745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:47.713186 5745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:18:47.713598 5745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:18:47.713663 5745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:47.713712 5745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:18:47.713747 5745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:47.720870 5745 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.67:46379
I20260810 07:18:47.720955 5886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.67:46379 every 8 connection(s)
I20260810 07:18:47.721369 5745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-2/data/info.pb
I20260810 07:18:47.727366 5437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:47.722253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.67:43257 (local address 127.15.217.126:39485)
0810 07:18:47.722326 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:47.722328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:47.722602 (+ 274us) server_negotiation.cc:407] Connection header received
0810 07:18:47.723177 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:47.723179 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:47.723193 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:47.723246 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:47.723978 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.724586 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.725776 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.725995 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.726075 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:47.727058 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:47.727059 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:47.727060 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:18:47.727066 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:47.727081 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:18:47.727085 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:47.727138 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:47.727242 (+ 104us) server_negotiation.cc:299] Negotiation successful
0810 07:18:47.727280 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 07:18:47.727615 5889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:47.722243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:39485 (local address 127.15.217.67:43257)
0810 07:18:47.722464 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:47.722501 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:47.723045 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:47.723321 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:47.723325 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:47.723333 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:18:47.723875 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:47.723880 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.724706 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.724707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:47.725681 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:47.725685 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.726107 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.726108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:47.726249 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:47.726889 (+ 640us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:18:47.726889 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:18:47.726921 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:18:47.726947 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:18:47.727145 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:18:47.727152 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:18:47.727156 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:18:47.727243 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 07:18:47.727354 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":109,"threads_started":1}
I20260810 07:18:47.728271 5887 heartbeater.cc:347] Connected to a master server at 127.15.217.126:39485
I20260810 07:18:47.728359 5887 heartbeater.cc:464] Registering TS with master...
I20260810 07:18:47.728616 5887 heartbeater.cc:511] Master 127.15.217.126:39485 requested a full tablet report, sending...
I20260810 07:18:47.729112 5393 ts_manager.cc:194] Registered new tserver with Master: 5db098d08e0b4c3d835c9bb46bb342b5 (127.15.217.67:46379)
I20260810 07:18:47.729559 5393 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.217.67:43257
I20260810 07:18:47.730885 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 5745
I20260810 07:18:47.730988 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-2/wal/instance
I20260810 07:18:47.731155 16229 external_mini_cluster.cc:1428] Setting key c778137b9fb662198a06ce46298e9878
I20260810 07:18:47.732498 16229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:18:47.736979 5437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:47.733954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34124 (local address 127.15.217.126:39485)
0810 07:18:47.734039 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:47.734041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:47.734146 (+ 105us) server_negotiation.cc:407] Connection header received
0810 07:18:47.734186 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:47.734186 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:47.734198 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:47.734248 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:47.734409 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.734896 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.735658 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.735850 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.735908 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:47.736691 (+ 783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:47.736692 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:47.736693 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:18:47.736697 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:47.736712 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:18:47.736716 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:47.736763 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:47.736830 (+ 67us) server_negotiation.cc:299] Negotiation successful
0810 07:18:47.736856 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 07:18:47.738775 5393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34124:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:18:47.740410 5393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:18:47.747464 5903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:47.743234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55608 (local address 127.15.217.66:40361)
0810 07:18:47.743483 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:47.743488 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:47.743507 (+ 19us) server_negotiation.cc:407] Connection header received
0810 07:18:47.743594 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:47.743597 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:47.743615 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:47.743701 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:47.743871 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.744808 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.746486 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.747018 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.747113 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:47.747219 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:47.747330 (+ 111us) server_negotiation.cc:299] Negotiation successful
0810 07:18:47.747373 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":120,"threads_started":1}
I20260810 07:18:47.747593 5898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:47.742882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:34141 (local address 127.0.0.1:47970)
0810 07:18:47.743213 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:47.743262 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:47.743336 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:47.744039 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:47.744040 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:47.744042 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:47.744082 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:47.744084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.745372 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.745373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:47.746230 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:47.746233 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.747222 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.747223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:47.747285 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:47.747290 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:18:47.747321 (+ 31us) client_negotiation.cc:240] Negotiation successful
0810 07:18:47.747451 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":25,"thread_start_us":67,"threads_started":1}
I20260810 07:18:47.747568 5902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:47.743105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.66:40361 (local address 127.0.0.1:55608)
0810 07:18:47.743440 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:47.743448 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:47.743483 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:47.743716 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:47.743719 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:47.743722 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:47.743800 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:47.743802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.744908 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.744909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:47.746393 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:47.746395 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.747126 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.747128 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:47.747222 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:47.747239 (+ 17us) client_negotiation.cc:769] Sending connection context
0810 07:18:47.747314 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 07:18:47.747445 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":25,"thread_start_us":85,"threads_started":1}
I20260810 07:18:47.747759 5901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:47.743006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34906 (local address 127.15.217.67:46379)
0810 07:18:47.743607 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:47.743612 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:47.743622 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:18:47.743664 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:47.743666 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:47.743683 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:47.743766 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:47.743951 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.745145 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.746466 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.747174 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.747235 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:47.747331 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:47.747618 (+ 287us) server_negotiation.cc:299] Negotiation successful
0810 07:18:47.747665 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":162,"threads_started":1}
I20260810 07:18:47.747777 5899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:47.742886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:46379 (local address 127.0.0.1:34906)
0810 07:18:47.743239 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:47.743283 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:47.743336 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:47.743786 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:47.743788 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:47.743791 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:47.743852 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:47.743853 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.745239 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.745240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:47.746367 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:47.746370 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.747365 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.747365 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:47.747461 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:47.747464 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:18:47.747519 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 07:18:47.747556 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"mutex_wait_us":31,"thread_start_us":101,"threads_started":1}
I20260810 07:18:47.748406 5900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:47.742980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47970 (local address 127.15.217.65:34141)
0810 07:18:47.743867 (+ 887us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:47.743872 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:47.743881 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:18:47.743923 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:47.743925 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:47.743941 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:47.744027 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:47.744154 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.745257 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.746335 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.747107 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.748040 (+ 933us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:47.748274 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:47.748307 (+ 33us) server_negotiation.cc:299] Negotiation successful
0810 07:18:47.748342 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":842,"thread_start_us":101,"threads_started":1}
I20260810 07:18:47.748744 5813 tablet_service.cc:1467] Processing CreateTablet for tablet 0344d7afb2c943b081a82534d3976899 (DEFAULT_TABLE table=test-table [id=8f4b3f304d464492a91ed482ca37efbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:18:47.749183 5813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0344d7afb2c943b081a82534d3976899. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:47.749145 5528 tablet_service.cc:1467] Processing CreateTablet for tablet 0344d7afb2c943b081a82534d3976899 (DEFAULT_TABLE table=test-table [id=8f4b3f304d464492a91ed482ca37efbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:18:47.749526 5528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0344d7afb2c943b081a82534d3976899. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:47.753198 5906 tablet_bootstrap.cc:492] T 0344d7afb2c943b081a82534d3976899 P 5db098d08e0b4c3d835c9bb46bb342b5: Bootstrap starting.
I20260810 07:18:47.753235 5673 tablet_service.cc:1467] Processing CreateTablet for tablet 0344d7afb2c943b081a82534d3976899 (DEFAULT_TABLE table=test-table [id=8f4b3f304d464492a91ed482ca37efbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:18:47.753633 5673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0344d7afb2c943b081a82534d3976899. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:47.753974 5906 tablet_bootstrap.cc:654] T 0344d7afb2c943b081a82534d3976899 P 5db098d08e0b4c3d835c9bb46bb342b5: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:47.754333 5906 log.cc:824] T 0344d7afb2c943b081a82534d3976899 P 5db098d08e0b4c3d835c9bb46bb342b5: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:47.754658 5905 tablet_bootstrap.cc:492] T 0344d7afb2c943b081a82534d3976899 P e06e739277014c43b9c77401cfe135bb: Bootstrap starting.
I20260810 07:18:47.755391 5905 tablet_bootstrap.cc:654] T 0344d7afb2c943b081a82534d3976899 P e06e739277014c43b9c77401cfe135bb: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:47.755748 5905 log.cc:824] T 0344d7afb2c943b081a82534d3976899 P e06e739277014c43b9c77401cfe135bb: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:47.756366 5906 tablet_bootstrap.cc:492] T 0344d7afb2c943b081a82534d3976899 P 5db098d08e0b4c3d835c9bb46bb342b5: No bootstrap required, opened a new log
I20260810 07:18:47.756446 5906 ts_tablet_manager.cc:1397] T 0344d7afb2c943b081a82534d3976899 P 5db098d08e0b4c3d835c9bb46bb342b5: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260810 07:18:47.756598 5905 tablet_bootstrap.cc:492] T 0344d7afb2c943b081a82534d3976899 P e06e739277014c43b9c77401cfe135bb: No bootstrap required, opened a new log
I20260810 07:18:47.756697 5905 ts_tablet_manager.cc:1397] T 0344d7afb2c943b081a82534d3976899 P e06e739277014c43b9c77401cfe135bb: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260810 07:18:47.757575 5909 tablet_bootstrap.cc:492] T 0344d7afb2c943b081a82534d3976899 P 93e547007e2141908f72433c7ababbf1: Bootstrap starting.
I20260810 07:18:47.758396 5909 tablet_bootstrap.cc:654] T 0344d7afb2c943b081a82534d3976899 P 93e547007e2141908f72433c7ababbf1: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:47.758790 5909 log.cc:824] T 0344d7afb2c943b081a82534d3976899 P 93e547007e2141908f72433c7ababbf1: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:47.759140 5906 raft_consensus.cc:348] T 0344d7afb2c943b081a82534d3976899 P 5db098d08e0b4c3d835c9bb46bb342b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db098d08e0b4c3d835c9bb46bb342b5" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 46379 } } peers { permanent_uuid: "e06e739277014c43b9c77401cfe135bb" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34141 } } peers { permanent_uuid: "93e547007e2141908f72433c7ababbf1" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 40361 } }
I20260810 07:18:47.759349 5906 raft_consensus.cc:374] T 0344d7afb2c943b081a82534d3976899 P 5db098d08e0b4c3d835c9bb46bb342b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:47.759393 5906 raft_consensus.cc:729] T 0344d7afb2c943b081a82534d3976899 P 5db098d08e0b4c3d835c9bb46bb342b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5db098d08e0b4c3d835c9bb46bb342b5, State: Initialized, Role: FOLLOWER
I20260810 07:18:47.759581 5906 consensus_queue.cc:260] T 0344d7afb2c943b081a82534d3976899 P 5db098d08e0b4c3d835c9bb46bb342b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db098d08e0b4c3d835c9bb46bb342b5" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 46379 } } peers { permanent_uuid: "e06e739277014c43b9c77401cfe135bb" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34141 } } peers { permanent_uuid: "93e547007e2141908f72433c7ababbf1" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 40361 } }
I20260810 07:18:47.759836 5909 tablet_bootstrap.cc:492] T 0344d7afb2c943b081a82534d3976899 P 93e547007e2141908f72433c7ababbf1: No bootstrap required, opened a new log
I20260810 07:18:47.759733 5905 raft_consensus.cc:348] T 0344d7afb2c943b081a82534d3976899 P e06e739277014c43b9c77401cfe135bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db098d08e0b4c3d835c9bb46bb342b5" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 46379 } } peers { permanent_uuid: "e06e739277014c43b9c77401cfe135bb" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34141 } } peers { permanent_uuid: "93e547007e2141908f72433c7ababbf1" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 40361 } }
I20260810 07:18:47.759886 5906 ts_tablet_manager.cc:1428] T 0344d7afb2c943b081a82534d3976899 P 5db098d08e0b4c3d835c9bb46bb342b5: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 07:18:47.759918 5909 ts_tablet_manager.cc:1397] T 0344d7afb2c943b081a82534d3976899 P 93e547007e2141908f72433c7ababbf1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:18:47.759924 5905 raft_consensus.cc:374] T 0344d7afb2c943b081a82534d3976899 P e06e739277014c43b9c77401cfe135bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:47.759966 5905 raft_consensus.cc:729] T 0344d7afb2c943b081a82534d3976899 P e06e739277014c43b9c77401cfe135bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e06e739277014c43b9c77401cfe135bb, State: Initialized, Role: FOLLOWER
I20260810 07:18:47.760025 5887 heartbeater.cc:503] Master 127.15.217.126:39485 was elected leader, sending a full tablet report...
I20260810 07:18:47.760115 5905 consensus_queue.cc:260] T 0344d7afb2c943b081a82534d3976899 P e06e739277014c43b9c77401cfe135bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db098d08e0b4c3d835c9bb46bb342b5" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 46379 } } peers { permanent_uuid: "e06e739277014c43b9c77401cfe135bb" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34141 } } peers { permanent_uuid: "93e547007e2141908f72433c7ababbf1" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 40361 } }
I20260810 07:18:47.760392 5905 ts_tablet_manager.cc:1428] T 0344d7afb2c943b081a82534d3976899 P e06e739277014c43b9c77401cfe135bb: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260810 07:18:47.760565 5601 heartbeater.cc:503] Master 127.15.217.126:39485 was elected leader, sending a full tablet report...
I20260810 07:18:47.762480 5909 raft_consensus.cc:348] T 0344d7afb2c943b081a82534d3976899 P 93e547007e2141908f72433c7ababbf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db098d08e0b4c3d835c9bb46bb342b5" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 46379 } } peers { permanent_uuid: "e06e739277014c43b9c77401cfe135bb" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34141 } } peers { permanent_uuid: "93e547007e2141908f72433c7ababbf1" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 40361 } }
I20260810 07:18:47.762625 5909 raft_consensus.cc:374] T 0344d7afb2c943b081a82534d3976899 P 93e547007e2141908f72433c7ababbf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:47.762650 5909 raft_consensus.cc:729] T 0344d7afb2c943b081a82534d3976899 P 93e547007e2141908f72433c7ababbf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93e547007e2141908f72433c7ababbf1, State: Initialized, Role: FOLLOWER
I20260810 07:18:47.762768 5909 consensus_queue.cc:260] T 0344d7afb2c943b081a82534d3976899 P 93e547007e2141908f72433c7ababbf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db098d08e0b4c3d835c9bb46bb342b5" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 46379 } } peers { permanent_uuid: "e06e739277014c43b9c77401cfe135bb" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34141 } } peers { permanent_uuid: "93e547007e2141908f72433c7ababbf1" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 40361 } }
I20260810 07:18:47.763010 5909 ts_tablet_manager.cc:1428] T 0344d7afb2c943b081a82534d3976899 P 93e547007e2141908f72433c7ababbf1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 07:18:47.763083 5742 heartbeater.cc:503] Master 127.15.217.126:39485 was elected leader, sending a full tablet report...
I20260810 07:18:47.836428 5913 raft_consensus.cc:482] T 0344d7afb2c943b081a82534d3976899 P 93e547007e2141908f72433c7ababbf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:18:47.836591 5913 raft_consensus.cc:504] T 0344d7afb2c943b081a82534d3976899 P 93e547007e2141908f72433c7ababbf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db098d08e0b4c3d835c9bb46bb342b5" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 46379 } } peers { permanent_uuid: "e06e739277014c43b9c77401cfe135bb" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34141 } } peers { permanent_uuid: "93e547007e2141908f72433c7ababbf1" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 40361 } }
I20260810 07:18:47.837069 5913 leader_election.cc:288] T 0344d7afb2c943b081a82534d3976899 P 93e547007e2141908f72433c7ababbf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5db098d08e0b4c3d835c9bb46bb342b5 (127.15.217.67:46379), e06e739277014c43b9c77401cfe135bb (127.15.217.65:34141)
W20260810 07:18:47.839462 5743 tablet.cc:2369] T 0344d7afb2c943b081a82534d3976899 P 93e547007e2141908f72433c7ababbf1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:18:47.840250 5901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:47.837417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:42415 (local address 127.15.217.67:46379)
0810 07:18:47.837511 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:47.837513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:47.837523 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:18:47.837557 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:47.837558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:47.837570 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:47.837673 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:47.837844 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.838363 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.839329 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.839843 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.839960 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:47.839970 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:47.840128 (+ 158us) server_negotiation.cc:299] Negotiation successful
0810 07:18:47.840172 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 07:18:47.840269 5744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:47.837284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.67:46379 (local address 127.15.217.66:42415)
0810 07:18:47.837354 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:47.837367 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:47.837428 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:47.837681 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:47.837682 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:47.837685 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:47.837765 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:47.837767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.838459 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.838460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:47.839221 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:47.839224 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.839961 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.839963 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:47.840040 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:47.840043 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:18:47.840108 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 07:18:47.840153 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260810 07:18:47.840268 5900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:47.837432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.217.66:48911 (local address 127.15.217.65:34141)
0810 07:18:47.837493 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:47.837495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:47.837638 (+ 143us) server_negotiation.cc:407] Connection header received
0810 07:18:47.837712 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:47.837713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:47.837726 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:47.837798 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:47.837967 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.838432 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.839399 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.839877 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.839972 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:47.839982 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:47.840155 (+ 173us) server_negotiation.cc:299] Negotiation successful
0810 07:18:47.840197 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 07:18:47.840292 5914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:47.837285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:34141 (local address 127.15.217.66:48911)
0810 07:18:47.837555 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:47.837564 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:47.837644 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:47.837805 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:47.837806 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:47.837808 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:47.837889 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:47.837890 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.838522 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.838523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:47.839299 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:47.839301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.839985 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.839986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:47.840066 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:47.840069 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:18:47.840136 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 07:18:47.840178 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":60,"thread_start_us":133,"threads_started":1}
I20260810 07:18:47.840672 5837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0344d7afb2c943b081a82534d3976899" candidate_uuid: "93e547007e2141908f72433c7ababbf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db098d08e0b4c3d835c9bb46bb342b5" is_pre_election: true
I20260810 07:18:47.840704 5551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0344d7afb2c943b081a82534d3976899" candidate_uuid: "93e547007e2141908f72433c7ababbf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e06e739277014c43b9c77401cfe135bb" is_pre_election: true
I20260810 07:18:47.840857 5837 raft_consensus.cc:2454] T 0344d7afb2c943b081a82534d3976899 P 5db098d08e0b4c3d835c9bb46bb342b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93e547007e2141908f72433c7ababbf1 in term 0.
I20260810 07:18:47.840864 5551 raft_consensus.cc:2454] T 0344d7afb2c943b081a82534d3976899 P e06e739277014c43b9c77401cfe135bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93e547007e2141908f72433c7ababbf1 in term 0.
I20260810 07:18:47.841128 5625 leader_election.cc:302] T 0344d7afb2c943b081a82534d3976899 P 93e547007e2141908f72433c7ababbf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93e547007e2141908f72433c7ababbf1, e06e739277014c43b9c77401cfe135bb; no voters:
I20260810 07:18:47.841317 5913 raft_consensus.cc:2781] T 0344d7afb2c943b081a82534d3976899 P 93e547007e2141908f72433c7ababbf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:18:47.841358 5913 raft_consensus.cc:482] T 0344d7afb2c943b081a82534d3976899 P 93e547007e2141908f72433c7ababbf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:18:47.841374 5913 raft_consensus.cc:3037] T 0344d7afb2c943b081a82534d3976899 P 93e547007e2141908f72433c7ababbf1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:47.842195 5913 raft_consensus.cc:504] T 0344d7afb2c943b081a82534d3976899 P 93e547007e2141908f72433c7ababbf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db098d08e0b4c3d835c9bb46bb342b5" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 46379 } } peers { permanent_uuid: "e06e739277014c43b9c77401cfe135bb" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34141 } } peers { permanent_uuid: "93e547007e2141908f72433c7ababbf1" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 40361 } }
I20260810 07:18:47.842343 5913 leader_election.cc:288] T 0344d7afb2c943b081a82534d3976899 P 93e547007e2141908f72433c7ababbf1 [CANDIDATE]: Term 1 election: Requested vote from peers 5db098d08e0b4c3d835c9bb46bb342b5 (127.15.217.67:46379), e06e739277014c43b9c77401cfe135bb (127.15.217.65:34141)
I20260810 07:18:47.842519 5837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0344d7afb2c943b081a82534d3976899" candidate_uuid: "93e547007e2141908f72433c7ababbf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5db098d08e0b4c3d835c9bb46bb342b5"
I20260810 07:18:47.842528 5551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0344d7afb2c943b081a82534d3976899" candidate_uuid: "93e547007e2141908f72433c7ababbf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e06e739277014c43b9c77401cfe135bb"
I20260810 07:18:47.842613 5837 raft_consensus.cc:3037] T 0344d7afb2c943b081a82534d3976899 P 5db098d08e0b4c3d835c9bb46bb342b5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:47.842619 5551 raft_consensus.cc:3037] T 0344d7afb2c943b081a82534d3976899 P e06e739277014c43b9c77401cfe135bb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:47.843580 5837 raft_consensus.cc:2454] T 0344d7afb2c943b081a82534d3976899 P 5db098d08e0b4c3d835c9bb46bb342b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93e547007e2141908f72433c7ababbf1 in term 1.
I20260810 07:18:47.843580 5551 raft_consensus.cc:2454] T 0344d7afb2c943b081a82534d3976899 P e06e739277014c43b9c77401cfe135bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93e547007e2141908f72433c7ababbf1 in term 1.
I20260810 07:18:47.843787 5625 leader_election.cc:302] T 0344d7afb2c943b081a82534d3976899 P 93e547007e2141908f72433c7ababbf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93e547007e2141908f72433c7ababbf1, e06e739277014c43b9c77401cfe135bb; no voters:
I20260810 07:18:47.843909 5913 raft_consensus.cc:2781] T 0344d7afb2c943b081a82534d3976899 P 93e547007e2141908f72433c7ababbf1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:18:47.844115 5913 raft_consensus.cc:686] T 0344d7afb2c943b081a82534d3976899 P 93e547007e2141908f72433c7ababbf1 [term 1 LEADER]: Becoming Leader. State: Replica: 93e547007e2141908f72433c7ababbf1, State: Running, Role: LEADER
I20260810 07:18:47.844224 5913 consensus_queue.cc:237] T 0344d7afb2c943b081a82534d3976899 P 93e547007e2141908f72433c7ababbf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db098d08e0b4c3d835c9bb46bb342b5" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 46379 } } peers { permanent_uuid: "e06e739277014c43b9c77401cfe135bb" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34141 } } peers { permanent_uuid: "93e547007e2141908f72433c7ababbf1" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 40361 } }
I20260810 07:18:47.845139 5393 catalog_manager.cc:5495] T 0344d7afb2c943b081a82534d3976899 P 93e547007e2141908f72433c7ababbf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 93e547007e2141908f72433c7ababbf1 (127.15.217.66). New cstate: current_term: 1 leader_uuid: "93e547007e2141908f72433c7ababbf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db098d08e0b4c3d835c9bb46bb342b5" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 46379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e06e739277014c43b9c77401cfe135bb" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93e547007e2141908f72433c7ababbf1" member_type: VOTER last_known_addr { host: "127.15.217.66" port: 40361 } health_report { overall_health: HEALTHY } } }
I20260810 07:18:47.864369 5903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:47.861702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55616 (local address 127.15.217.66:40361)
0810 07:18:47.861773 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:47.861775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:47.861786 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:18:47.861884 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:47.861885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:47.861906 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:47.861992 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:18:47.862152 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.862638 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.863500 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:47.863699 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:47.863777 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:47.863971 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:18:47.864095 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:18:47.864199 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:47.864225 (+ 26us) server_negotiation.cc:299] Negotiation successful
0810 07:18:47.864244 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 07:18:47.866561 5837 raft_consensus.cc:1261] T 0344d7afb2c943b081a82534d3976899 P 5db098d08e0b4c3d835c9bb46bb342b5 [term 1 FOLLOWER]: Refusing update from remote peer 93e547007e2141908f72433c7ababbf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:18:47.866561 5551 raft_consensus.cc:1261] T 0344d7afb2c943b081a82534d3976899 P e06e739277014c43b9c77401cfe135bb [term 1 FOLLOWER]: Refusing update from remote peer 93e547007e2141908f72433c7ababbf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:18:47.866989 5915 consensus_queue.cc:1035] T 0344d7afb2c943b081a82534d3976899 P 93e547007e2141908f72433c7ababbf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5db098d08e0b4c3d835c9bb46bb342b5" member_type: VOTER last_known_addr { host: "127.15.217.67" port: 46379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:18:47.867161 5913 consensus_queue.cc:1035] T 0344d7afb2c943b081a82534d3976899 P 93e547007e2141908f72433c7ababbf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e06e739277014c43b9c77401cfe135bb" member_type: VOTER last_known_addr { host: "127.15.217.65" port: 34141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:18:47.869972 5918 mvcc.cc:205] Tried to move back new op lower bound from 7316874558938353664 to 7316874558850560000. Current Snapshot: MvccSnapshot[applied={T|T < 7316874558938353664}]
I20260810 07:18:47.870229 5919 mvcc.cc:205] Tried to move back new op lower bound from 7316874558938353664 to 7316874558850560000. Current Snapshot: MvccSnapshot[applied={T|T < 7316874558938353664}]
I20260810 07:18:47.870918 5920 mvcc.cc:205] Tried to move back new op lower bound from 7316874558938353664 to 7316874558850560000. Current Snapshot: MvccSnapshot[applied={T|T < 7316874558938353664}]
I20260810 07:18:47.872334 5392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34124:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:18:47.872483 5392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34124:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:18:47.873297 5392 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0b4f73098624b629c54e18095eb2ae5: Sending DeleteTablet for 3 replicas of tablet 0344d7afb2c943b081a82534d3976899
I20260810 07:18:47.873664 5673 tablet_service.cc:1514] Processing DeleteTablet for tablet 0344d7afb2c943b081a82534d3976899 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:18:47 UTC) from {username='slave'} at 127.0.0.1:55608
I20260810 07:18:47.873695 5813 tablet_service.cc:1514] Processing DeleteTablet for tablet 0344d7afb2c943b081a82534d3976899 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:18:47 UTC) from {username='slave'} at 127.0.0.1:34906
I20260810 07:18:47.873690 5528 tablet_service.cc:1514] Processing DeleteTablet for tablet 0344d7afb2c943b081a82534d3976899 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:18:47 UTC) from {username='slave'} at 127.0.0.1:47970
I20260810 07:18:47.874312 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 5461
I20260810 07:18:47.876698 5929 tablet_replica.cc:330] T 0344d7afb2c943b081a82534d3976899 P 93e547007e2141908f72433c7ababbf1: stopping tablet replica
I20260810 07:18:47.876791 5929 raft_consensus.cc:2229] T 0344d7afb2c943b081a82534d3976899 P 93e547007e2141908f72433c7ababbf1 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:18:47.876891 5929 raft_consensus.cc:2258] T 0344d7afb2c943b081a82534d3976899 P 93e547007e2141908f72433c7ababbf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:18:47.877110 5930 tablet_replica.cc:330] T 0344d7afb2c943b081a82534d3976899 P 5db098d08e0b4c3d835c9bb46bb342b5: stopping tablet replica
I20260810 07:18:47.877183 5930 raft_consensus.cc:2229] T 0344d7afb2c943b081a82534d3976899 P 5db098d08e0b4c3d835c9bb46bb342b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:18:47.877250 5930 raft_consensus.cc:2258] T 0344d7afb2c943b081a82534d3976899 P 5db098d08e0b4c3d835c9bb46bb342b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:18:47.877718 5930 ts_tablet_manager.cc:1905] T 0344d7afb2c943b081a82534d3976899 P 5db098d08e0b4c3d835c9bb46bb342b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:18:47.879357 5930 ts_tablet_manager.cc:1918] T 0344d7afb2c943b081a82534d3976899 P 5db098d08e0b4c3d835c9bb46bb342b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:18:47.879437 5930 log.cc:1196] T 0344d7afb2c943b081a82534d3976899 P 5db098d08e0b4c3d835c9bb46bb342b5: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-2/wal/wals/0344d7afb2c943b081a82534d3976899
I20260810 07:18:47.879657 5929 ts_tablet_manager.cc:1905] T 0344d7afb2c943b081a82534d3976899 P 93e547007e2141908f72433c7ababbf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:18:47.879724 5930 ts_tablet_manager.cc:1939] T 0344d7afb2c943b081a82534d3976899 P 5db098d08e0b4c3d835c9bb46bb342b5: Deleting consensus metadata
I20260810 07:18:47.880187 5377 catalog_manager.cc:4841] TS 5db098d08e0b4c3d835c9bb46bb342b5 (127.15.217.67:46379): tablet 0344d7afb2c943b081a82534d3976899 (table test-table [id=8f4b3f304d464492a91ed482ca37efbd]) successfully deleted
I20260810 07:18:47.881877 5929 ts_tablet_manager.cc:1918] T 0344d7afb2c943b081a82534d3976899 P 93e547007e2141908f72433c7ababbf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:18:47.881966 5929 log.cc:1196] T 0344d7afb2c943b081a82534d3976899 P 93e547007e2141908f72433c7ababbf1: Deleting WAL directory at /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346081759552-16229-0/minicluster-data/ts-1/wal/wals/0344d7afb2c943b081a82534d3976899
I20260810 07:18:47.882242 5929 ts_tablet_manager.cc:1939] T 0344d7afb2c943b081a82534d3976899 P 93e547007e2141908f72433c7ababbf1: Deleting consensus metadata
I20260810 07:18:47.882738 5377 catalog_manager.cc:4841] TS 93e547007e2141908f72433c7ababbf1 (127.15.217.66:40361): tablet 0344d7afb2c943b081a82534d3976899 (table test-table [id=8f4b3f304d464492a91ed482ca37efbd]) successfully deleted
W20260810 07:18:47.886965 5379 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.217.65:34141 (error 108)
I20260810 07:18:47.887681 5899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:47.887512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.65:34141 (local address 127.0.0.1:47982)
0810 07:18:47.887587 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:47.887645 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.65:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 07:18:47.887662 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 5605
W20260810 07:18:47.889628 5379 catalog_manager.cc:4584] TS e06e739277014c43b9c77401cfe135bb (127.15.217.65:34141): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0344d7afb2c943b081a82534d3976899: Network error: Client connection negotiation failed: client connection to 127.15.217.65:34141: connect: Connection refused (error 111)
I20260810 07:18:47.895632 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 5745
I20260810 07:18:47.902789 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 5361
2026-08-10T07:18:47Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (799 ms)
[----------] 6 tests from EncryptionPolicyTest (9245 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:18:47 dist-test-slave-10pc krb5kdc[5935](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:18:47 dist-test-slave-10pc krb5kdc[5935](info): set up 2 sockets
Aug 10 07:18:47 dist-test-slave-10pc krb5kdc[5935](info): commencing operation
krb5kdc: starting...
W20260810 07:18:49.013207 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:18:49 dist-test-slave-10pc krb5kdc[5935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:18:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:18:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:18:49.151516 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37835
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37835
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:18:49.240403 5948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:49.240662 5948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:49.240686 5948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:49.241880 5948 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:18:49.241927 5948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:49.241941 5948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:49.241954 5948 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:18:49.241967 5948 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:18:49.243516 5948 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37835
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:49.243814 5948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:49.244081 5948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:49.249329 5954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:49.249344 5956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:49.249329 5953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:49.249691 5948 server_base.cc:1034] running on GCE node
I20260810 07:18:49.249976 5948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:49.250308 5948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:49.251502 5948 hybrid_clock.cc:648] HybridClock initialized: now 1786346329251482 us; error 33 us; skew 500 ppm
Aug 10 07:18:49 dist-test-slave-10pc krb5kdc[5935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346329, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:49.259152 5948 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:18:49.259739 5948 webserver.cc:460] Webserver started at http://127.0.0.1:43791/ using document root <none> and password file <none>
I20260810 07:18:49.260048 5948 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:49.260134 5948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:49.260320 5948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:49.261818 5948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "aa3650b86e4a4c61a75ee50120e03729"
format_stamp: "Formatted at 2026-08-10 07:18:49 on dist-test-slave-10pc"
server_key: "0adec7a4a29205b8bb5b0cdbb2b47044"
server_key_iv: "ee2f2dd3c154cbb625044f2e7e9d59ca"
server_key_version: "encryptionkey@0"
I20260810 07:18:49.262298 5948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "aa3650b86e4a4c61a75ee50120e03729"
format_stamp: "Formatted at 2026-08-10 07:18:49 on dist-test-slave-10pc"
server_key: "0adec7a4a29205b8bb5b0cdbb2b47044"
server_key_iv: "ee2f2dd3c154cbb625044f2e7e9d59ca"
server_key_version: "encryptionkey@0"
I20260810 07:18:49.264015 5948 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:18:49.265177 5963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:49.265450 5948 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:18:49.265587 5948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "aa3650b86e4a4c61a75ee50120e03729"
format_stamp: "Formatted at 2026-08-10 07:18:49 on dist-test-slave-10pc"
server_key: "0adec7a4a29205b8bb5b0cdbb2b47044"
server_key_iv: "ee2f2dd3c154cbb625044f2e7e9d59ca"
server_key_version: "encryptionkey@0"
I20260810 07:18:49.265678 5948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:49.286751 5948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:49.289392 5948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:49.289599 5948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:49.294194 5948 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37835
I20260810 07:18:49.294261 6020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37835 every 8 connection(s)
I20260810 07:18:49.294682 5948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:18:49.295704 6021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:49.298471 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 5948
I20260810 07:18:49.298635 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:18:49.298580 6021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa3650b86e4a4c61a75ee50120e03729: Bootstrap starting.
I20260810 07:18:49.298848 16229 external_mini_cluster.cc:1428] Setting key 20f4ed8e88b82f92917126f1989e5a6e
I20260810 07:18:49.299481 6021 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa3650b86e4a4c61a75ee50120e03729: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:49.299881 6021 log.cc:824] T 00000000000000000000000000000000 P aa3650b86e4a4c61a75ee50120e03729: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:49.300846 6021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa3650b86e4a4c61a75ee50120e03729: No bootstrap required, opened a new log
I20260810 07:18:49.303480 6021 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa3650b86e4a4c61a75ee50120e03729 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3650b86e4a4c61a75ee50120e03729" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37835 } }
I20260810 07:18:49.303685 6021 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa3650b86e4a4c61a75ee50120e03729 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:49.303723 6021 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa3650b86e4a4c61a75ee50120e03729 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa3650b86e4a4c61a75ee50120e03729, State: Initialized, Role: FOLLOWER
I20260810 07:18:49.303884 6021 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa3650b86e4a4c61a75ee50120e03729 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3650b86e4a4c61a75ee50120e03729" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37835 } }
I20260810 07:18:49.303995 6021 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa3650b86e4a4c61a75ee50120e03729 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:18:49.304028 6021 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa3650b86e4a4c61a75ee50120e03729 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:18:49.304080 6021 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa3650b86e4a4c61a75ee50120e03729 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:49.305045 6021 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa3650b86e4a4c61a75ee50120e03729 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3650b86e4a4c61a75ee50120e03729" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37835 } }
I20260810 07:18:49.305181 6021 leader_election.cc:302] T 00000000000000000000000000000000 P aa3650b86e4a4c61a75ee50120e03729 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa3650b86e4a4c61a75ee50120e03729; no voters:
I20260810 07:18:49.305380 6021 leader_election.cc:288] T 00000000000000000000000000000000 P aa3650b86e4a4c61a75ee50120e03729 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:18:49.305451 6026 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa3650b86e4a4c61a75ee50120e03729 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:18:49.305645 6026 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa3650b86e4a4c61a75ee50120e03729 [term 1 LEADER]: Becoming Leader. State: Replica: aa3650b86e4a4c61a75ee50120e03729, State: Running, Role: LEADER
I20260810 07:18:49.305739 6021 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa3650b86e4a4c61a75ee50120e03729 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:18:49.305882 6026 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa3650b86e4a4c61a75ee50120e03729 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3650b86e4a4c61a75ee50120e03729" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37835 } }
I20260810 07:18:49.306389 6026 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa3650b86e4a4c61a75ee50120e03729 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa3650b86e4a4c61a75ee50120e03729. Latest consensus state: current_term: 1 leader_uuid: "aa3650b86e4a4c61a75ee50120e03729" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3650b86e4a4c61a75ee50120e03729" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37835 } } }
I20260810 07:18:49.306553 6026 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa3650b86e4a4c61a75ee50120e03729 [sys.catalog]: This master's current role is: LEADER
I20260810 07:18:49.306859 6036 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 10 07:18:49 dist-test-slave-10pc krb5kdc[5935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 07:18:49.307538 6036 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:18:49.307786 6028 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa3650b86e4a4c61a75ee50120e03729 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa3650b86e4a4c61a75ee50120e03729" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3650b86e4a4c61a75ee50120e03729" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37835 } } }
I20260810 07:18:49.307912 6028 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa3650b86e4a4c61a75ee50120e03729 [sys.catalog]: This master's current role is: LEADER
I20260810 07:18:49.310065 6036 catalog_manager.cc:1347] Generated new cluster ID: ee022f08dbc141adbd6e8d12fb8ce79b
I20260810 07:18:49.310124 6036 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:18:49.312377 6024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:49.299579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46999 (local address 127.0.0.1:37835)
0810 07:18:49.299807 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:49.299816 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:49.299854 (+ 38us) server_negotiation.cc:407] Connection header received
0810 07:18:49.300438 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:49.300448 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:49.300566 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:49.300691 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:49.301612 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:49.302553 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:49.303851 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:49.304603 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:49.304730 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:49.308095 (+ 3365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:49.308102 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:49.308102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:49.308120 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:49.310571 (+ 2451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:49.311066 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:49.311069 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:49.311070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:49.311172 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:49.311447 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:49.311448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:49.311449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:49.311737 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:49.311880 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:49.312005 (+ 125us) server_negotiation.cc:299] Negotiation successful
0810 07:18:49.312144 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":90,"threads_started":1}
I20260810 07:18:49.313557 16229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:18:49.323333 6024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:49.315105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54739 (local address 127.0.0.1:37835)
0810 07:18:49.315155 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:49.315157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:49.315338 (+ 181us) server_negotiation.cc:407] Connection header received
0810 07:18:49.315385 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:49.315386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:49.315397 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:49.315465 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:49.316108 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:49.316946 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:49.318000 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:49.318353 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:49.318439 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:49.320020 (+ 1581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:49.320021 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:49.320022 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:49.320029 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:49.321851 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:49.322315 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:49.322317 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:49.322317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:49.322394 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:49.322703 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:49.322704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:49.322705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:49.322854 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:49.322967 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:49.323145 (+ 178us) server_negotiation.cc:299] Negotiation successful
0810 07:18:49.323220 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 07:18:49.325047 6036 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:18:49.325716 6036 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:18:49.342130 6036 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa3650b86e4a4c61a75ee50120e03729: Generated new TSK 0
I20260810 07:18:49.342404 6036 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:18:49.345752 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 5948
W20260810 07:18:49.354077 5933 connection.cc:535] client connection to 127.0.0.1:37835 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37835): Connection reset by peer (error 104)
2026-08-10T07:18:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1441 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:18:49 dist-test-slave-10pc krb5kdc[6055](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:18:49 dist-test-slave-10pc krb5kdc[6055](info): set up 2 sockets
Aug 10 07:18:49 dist-test-slave-10pc krb5kdc[6055](info): commencing operation
krb5kdc: starting...
W20260810 07:18:50.460711 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:18:50 dist-test-slave-10pc krb5kdc[6055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:18:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:18:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:18:50.599635 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41501
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41501
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:18:50.689252 6069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:50.689492 6069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:50.689518 6069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:50.690812 6069 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:18:50.690876 6069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:50.690891 6069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:50.690903 6069 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:18:50.690917 6069 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:18:50.692643 6069 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41501
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:50.692960 6069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:50.693235 6069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:50.698724 6074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:50.698758 6069 server_base.cc:1034] running on GCE node
W20260810 07:18:50.698724 6077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:50.698721 6075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:50.699376 6069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:50.699682 6069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:50.700836 6069 hybrid_clock.cc:648] HybridClock initialized: now 1786346330700820 us; error 30 us; skew 500 ppm
Aug 10 07:18:50 dist-test-slave-10pc krb5kdc[6055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346330, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:50.703733 6069 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:18:50.704247 6069 webserver.cc:460] Webserver started at http://127.0.0.1:35293/ using document root <none> and password file <none>
I20260810 07:18:50.704546 6069 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:50.704602 6069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:50.704703 6069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:50.705859 6069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "704b09adb35a42d6bb3b494d41df9f21"
format_stamp: "Formatted at 2026-08-10 07:18:50 on dist-test-slave-10pc"
server_key: "5891d7480a020f5100a0638f57be2ecd"
server_key_iv: "c0d31d9d95ce94c3bde86dd4c09f94e0"
server_key_version: "encryptionkey@0"
I20260810 07:18:50.706246 6069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "704b09adb35a42d6bb3b494d41df9f21"
format_stamp: "Formatted at 2026-08-10 07:18:50 on dist-test-slave-10pc"
server_key: "5891d7480a020f5100a0638f57be2ecd"
server_key_iv: "c0d31d9d95ce94c3bde86dd4c09f94e0"
server_key_version: "encryptionkey@0"
I20260810 07:18:50.707626 6069 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:18:50.708400 6085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:50.708657 6069 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:18:50.708739 6069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "704b09adb35a42d6bb3b494d41df9f21"
format_stamp: "Formatted at 2026-08-10 07:18:50 on dist-test-slave-10pc"
server_key: "5891d7480a020f5100a0638f57be2ecd"
server_key_iv: "c0d31d9d95ce94c3bde86dd4c09f94e0"
server_key_version: "encryptionkey@0"
I20260810 07:18:50.708838 6069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:50.731457 6069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:50.734203 6069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:50.734386 6069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:50.738972 6069 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41501
I20260810 07:18:50.739018 6143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41501 every 8 connection(s)
I20260810 07:18:50.739445 6069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:18:50.740388 6144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:50.742858 6144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 704b09adb35a42d6bb3b494d41df9f21: Bootstrap starting.
I20260810 07:18:50.743454 6144 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 704b09adb35a42d6bb3b494d41df9f21: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:50.743717 6144 log.cc:824] T 00000000000000000000000000000000 P 704b09adb35a42d6bb3b494d41df9f21: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:50.744491 6144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 704b09adb35a42d6bb3b494d41df9f21: No bootstrap required, opened a new log
I20260810 07:18:50.746217 6144 raft_consensus.cc:348] T 00000000000000000000000000000000 P 704b09adb35a42d6bb3b494d41df9f21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "704b09adb35a42d6bb3b494d41df9f21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41501 } }
I20260810 07:18:50.746373 6144 raft_consensus.cc:374] T 00000000000000000000000000000000 P 704b09adb35a42d6bb3b494d41df9f21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:50.746397 6144 raft_consensus.cc:729] T 00000000000000000000000000000000 P 704b09adb35a42d6bb3b494d41df9f21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 704b09adb35a42d6bb3b494d41df9f21, State: Initialized, Role: FOLLOWER
I20260810 07:18:50.746554 6144 consensus_queue.cc:260] T 00000000000000000000000000000000 P 704b09adb35a42d6bb3b494d41df9f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "704b09adb35a42d6bb3b494d41df9f21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41501 } }
I20260810 07:18:50.746654 6144 raft_consensus.cc:388] T 00000000000000000000000000000000 P 704b09adb35a42d6bb3b494d41df9f21 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:18:50.746681 6144 raft_consensus.cc:482] T 00000000000000000000000000000000 P 704b09adb35a42d6bb3b494d41df9f21 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:18:50.746740 6144 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 704b09adb35a42d6bb3b494d41df9f21 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:50.746891 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 6069
I20260810 07:18:50.747007 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:18:50.747242 16229 external_mini_cluster.cc:1428] Setting key 72bbfd622028257b2a8a49a57d9404e7
I20260810 07:18:50.747498 6144 raft_consensus.cc:504] T 00000000000000000000000000000000 P 704b09adb35a42d6bb3b494d41df9f21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "704b09adb35a42d6bb3b494d41df9f21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41501 } }
I20260810 07:18:50.747642 6144 leader_election.cc:302] T 00000000000000000000000000000000 P 704b09adb35a42d6bb3b494d41df9f21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 704b09adb35a42d6bb3b494d41df9f21; no voters:
I20260810 07:18:50.747830 6144 leader_election.cc:288] T 00000000000000000000000000000000 P 704b09adb35a42d6bb3b494d41df9f21 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:18:50.747864 6148 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 704b09adb35a42d6bb3b494d41df9f21 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:18:50.748153 6148 raft_consensus.cc:686] T 00000000000000000000000000000000 P 704b09adb35a42d6bb3b494d41df9f21 [term 1 LEADER]: Becoming Leader. State: Replica: 704b09adb35a42d6bb3b494d41df9f21, State: Running, Role: LEADER
I20260810 07:18:50.748291 6144 sys_catalog.cc:564] T 00000000000000000000000000000000 P 704b09adb35a42d6bb3b494d41df9f21 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:18:50.748298 6148 consensus_queue.cc:237] T 00000000000000000000000000000000 P 704b09adb35a42d6bb3b494d41df9f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "704b09adb35a42d6bb3b494d41df9f21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41501 } }
I20260810 07:18:50.748911 6150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 704b09adb35a42d6bb3b494d41df9f21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "704b09adb35a42d6bb3b494d41df9f21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "704b09adb35a42d6bb3b494d41df9f21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41501 } } }
I20260810 07:18:50.749022 6150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 704b09adb35a42d6bb3b494d41df9f21 [sys.catalog]: This master's current role is: LEADER
I20260810 07:18:50.749224 6151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 704b09adb35a42d6bb3b494d41df9f21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 704b09adb35a42d6bb3b494d41df9f21. Latest consensus state: current_term: 1 leader_uuid: "704b09adb35a42d6bb3b494d41df9f21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "704b09adb35a42d6bb3b494d41df9f21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41501 } } }
I20260810 07:18:50.749454 6159 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:18:50.749500 6151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 704b09adb35a42d6bb3b494d41df9f21 [sys.catalog]: This master's current role is: LEADER
I20260810 07:18:50.750106 6159 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:18:50.751968 6159 catalog_manager.cc:1347] Generated new cluster ID: c78d9065c4434f348d7644067a2d6c97
I20260810 07:18:50.752029 6159 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:18:50.754000 6149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:50.747866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58753 (local address 127.0.0.1:41501)
0810 07:18:50.748171 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:50.748181 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:50.748215 (+ 34us) server_negotiation.cc:407] Connection header received
0810 07:18:50.748347 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:50.748355 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:50.748418 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:50.748637 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:50.748954 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:50.750649 (+ 1695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:50.751778 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:50.752373 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:50.752531 (+ 158us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:50.753341 (+ 810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:50.753345 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:50.753346 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:18:50.753533 (+ 187us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:50.753572 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:18:50.753584 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:50.753683 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:50.753777 (+ 94us) server_negotiation.cc:299] Negotiation successful
0810 07:18:50.753808 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":110,"threads_started":1}
I20260810 07:18:50.757665 16229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:18:50.762208 6159 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:18:50.762754 6149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:50.759202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33215 (local address 127.0.0.1:41501)
0810 07:18:50.759263 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:50.759265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:50.759383 (+ 118us) server_negotiation.cc:407] Connection header received
0810 07:18:50.759472 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:50.759473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:50.759483 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:50.759539 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:50.759718 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:50.760277 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:50.761118 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:50.761374 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:50.761463 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:50.762450 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:50.762452 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:50.762452 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:18:50.762458 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:50.762475 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:18:50.762480 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:50.762574 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:50.762638 (+ 64us) server_negotiation.cc:299] Negotiation successful
0810 07:18:50.762650 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 07:18:50.762970 6159 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:18:50.773082 6159 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 704b09adb35a42d6bb3b494d41df9f21: Generated new TSK 0
I20260810 07:18:50.773371 6159 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:18:50.782213 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 6069
W20260810 07:18:50.790556 6053 connection.cc:535] client connection to 127.0.0.1:41501 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41501): Connection reset by peer (error 104)
2026-08-10T07:18:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1436 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:18:50 dist-test-slave-10pc krb5kdc[6176](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:18:50 dist-test-slave-10pc krb5kdc[6176](info): set up 2 sockets
Aug 10 07:18:50 dist-test-slave-10pc krb5kdc[6176](info): commencing operation
krb5kdc: starting...
W20260810 07:18:51.888625 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.007s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:18:51 dist-test-slave-10pc krb5kdc[6176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346331, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:18:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:18:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:18:52.027992 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46779
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46779
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:18:52.116286 6191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:52.116566 6191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:52.116608 6191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:52.117776 6191 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:18:52.117848 6191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:52.117873 6191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:52.117901 6191 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:18:52.117929 6191 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:18:52.119481 6191 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46779
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:52.119778 6191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:52.120024 6191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:52.125144 6199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:52.125159 6196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:52.125232 6191 server_base.cc:1034] running on GCE node
W20260810 07:18:52.125334 6197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:52.125694 6191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:52.125948 6191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:52.127120 6191 hybrid_clock.cc:648] HybridClock initialized: now 1786346332127112 us; error 54 us; skew 500 ppm
Aug 10 07:18:52 dist-test-slave-10pc krb5kdc[6176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346332, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:52.129757 6191 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:18:52.130240 6191 webserver.cc:460] Webserver started at http://127.0.0.1:35755/ using document root <none> and password file <none>
I20260810 07:18:52.130450 6191 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:52.130493 6191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:52.130589 6191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:52.131661 6191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "d6aa0c0084b24b76992c0189c0b60d88"
format_stamp: "Formatted at 2026-08-10 07:18:52 on dist-test-slave-10pc"
server_key: "172ee622135066ab53867adf930b931e"
server_key_iv: "e008875b25150477d40c7e4ee2a9f6e7"
server_key_version: "encryptionkey@0"
I20260810 07:18:52.131994 6191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "d6aa0c0084b24b76992c0189c0b60d88"
format_stamp: "Formatted at 2026-08-10 07:18:52 on dist-test-slave-10pc"
server_key: "172ee622135066ab53867adf930b931e"
server_key_iv: "e008875b25150477d40c7e4ee2a9f6e7"
server_key_version: "encryptionkey@0"
I20260810 07:18:52.133494 6191 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:18:52.134284 6206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:52.134497 6191 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:18:52.134579 6191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "d6aa0c0084b24b76992c0189c0b60d88"
format_stamp: "Formatted at 2026-08-10 07:18:52 on dist-test-slave-10pc"
server_key: "172ee622135066ab53867adf930b931e"
server_key_iv: "e008875b25150477d40c7e4ee2a9f6e7"
server_key_version: "encryptionkey@0"
I20260810 07:18:52.134683 6191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:52.138196 6191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:52.140331 6191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:52.140513 6191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:52.144979 6191 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46779
I20260810 07:18:52.145015 6265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46779 every 8 connection(s)
I20260810 07:18:52.145368 6191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:18:52.146312 6266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:52.148753 6266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6aa0c0084b24b76992c0189c0b60d88: Bootstrap starting.
I20260810 07:18:52.149374 6266 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6aa0c0084b24b76992c0189c0b60d88: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:52.149648 6266 log.cc:824] T 00000000000000000000000000000000 P d6aa0c0084b24b76992c0189c0b60d88: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:52.150377 6266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6aa0c0084b24b76992c0189c0b60d88: No bootstrap required, opened a new log
I20260810 07:18:52.152076 6266 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6aa0c0084b24b76992c0189c0b60d88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6aa0c0084b24b76992c0189c0b60d88" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46779 } }
I20260810 07:18:52.152233 6266 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6aa0c0084b24b76992c0189c0b60d88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:52.152256 6266 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6aa0c0084b24b76992c0189c0b60d88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6aa0c0084b24b76992c0189c0b60d88, State: Initialized, Role: FOLLOWER
I20260810 07:18:52.152369 6266 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6aa0c0084b24b76992c0189c0b60d88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6aa0c0084b24b76992c0189c0b60d88" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46779 } }
I20260810 07:18:52.152427 6266 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6aa0c0084b24b76992c0189c0b60d88 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:18:52.152513 6266 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6aa0c0084b24b76992c0189c0b60d88 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:18:52.152567 6266 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6aa0c0084b24b76992c0189c0b60d88 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:52.153133 6266 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6aa0c0084b24b76992c0189c0b60d88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6aa0c0084b24b76992c0189c0b60d88" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46779 } }
I20260810 07:18:52.153235 6266 leader_election.cc:302] T 00000000000000000000000000000000 P d6aa0c0084b24b76992c0189c0b60d88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6aa0c0084b24b76992c0189c0b60d88; no voters:
I20260810 07:18:52.153376 6266 leader_election.cc:288] T 00000000000000000000000000000000 P d6aa0c0084b24b76992c0189c0b60d88 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:18:52.153455 6269 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6aa0c0084b24b76992c0189c0b60d88 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:18:52.153644 6269 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6aa0c0084b24b76992c0189c0b60d88 [term 1 LEADER]: Becoming Leader. State: Replica: d6aa0c0084b24b76992c0189c0b60d88, State: Running, Role: LEADER
I20260810 07:18:52.153693 6266 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6aa0c0084b24b76992c0189c0b60d88 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:18:52.153749 6269 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6aa0c0084b24b76992c0189c0b60d88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6aa0c0084b24b76992c0189c0b60d88" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46779 } }
I20260810 07:18:52.154191 6271 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6aa0c0084b24b76992c0189c0b60d88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6aa0c0084b24b76992c0189c0b60d88. Latest consensus state: current_term: 1 leader_uuid: "d6aa0c0084b24b76992c0189c0b60d88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6aa0c0084b24b76992c0189c0b60d88" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46779 } } }
I20260810 07:18:52.154280 6271 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6aa0c0084b24b76992c0189c0b60d88 [sys.catalog]: This master's current role is: LEADER
I20260810 07:18:52.154321 6270 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6aa0c0084b24b76992c0189c0b60d88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6aa0c0084b24b76992c0189c0b60d88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6aa0c0084b24b76992c0189c0b60d88" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46779 } } }
I20260810 07:18:52.154383 6270 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6aa0c0084b24b76992c0189c0b60d88 [sys.catalog]: This master's current role is: LEADER
I20260810 07:18:52.154388 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 6191
I20260810 07:18:52.154513 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:18:52.154506 6273 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:18:52.154742 16229 external_mini_cluster.cc:1428] Setting key 3d04cc08397a4c8179ac50f5b921b934
I20260810 07:18:52.155113 6273 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:18:52.156993 6273 catalog_manager.cc:1347] Generated new cluster ID: 60e4cc0d0b44499ea65947e2bde7b9f6
I20260810 07:18:52.157053 6273 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:18:52.157860 6288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:52.155489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43789 (local address 127.0.0.1:46779)
0810 07:18:52.155916 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:52.155927 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:52.155961 (+ 34us) server_negotiation.cc:407] Connection header received
0810 07:18:52.156088 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:52.156095 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:52.156127 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:52.156258 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:52.157306 (+ 1048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:52.157310 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:52.157310 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:18:52.157314 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:52.157359 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:18:52.157373 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:52.157437 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:52.157542 (+ 105us) server_negotiation.cc:299] Negotiation successful
0810 07:18:52.157676 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":93,"threads_started":1}
I20260810 07:18:52.158592 16229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:18:52.162112 6288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:52.160874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52327 (local address 127.0.0.1:46779)
0810 07:18:52.160914 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:52.160916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:52.161138 (+ 222us) server_negotiation.cc:407] Connection header received
0810 07:18:52.161199 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:52.161199 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:52.161205 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:52.161257 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:52.161890 (+ 633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:52.161892 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:52.161892 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:18:52.161896 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:52.161910 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:18:52.161914 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:52.161968 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:52.162036 (+ 68us) server_negotiation.cc:299] Negotiation successful
0810 07:18:52.162050 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 07:18:52.166801 6273 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:18:52.167038 6273 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:18:52.177875 6273 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6aa0c0084b24b76992c0189c0b60d88: Generated new TSK 0
I20260810 07:18:52.178134 6273 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:18:52.182729 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 6191
2026-08-10T07:18:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1400 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:18:52 dist-test-slave-10pc krb5kdc[6301](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:18:52 dist-test-slave-10pc krb5kdc[6301](info): set up 2 sockets
Aug 10 07:18:52 dist-test-slave-10pc krb5kdc[6301](info): commencing operation
krb5kdc: starting...
W20260810 07:18:53.288340 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:18:53 dist-test-slave-10pc krb5kdc[6301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:18:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:18:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:18:53.428179 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46793
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46793
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:18:53.516088 6315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:53.516347 6315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:53.516390 6315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:53.517616 6315 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:18:53.517685 6315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:53.517709 6315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:53.517729 6315 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:18:53.517760 6315 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:18:53.519312 6315 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:53.519620 6315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:53.519871 6315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:53.525017 6320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:53.525136 6315 server_base.cc:1034] running on GCE node
W20260810 07:18:53.525023 6321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:53.525034 6323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:53.525552 6315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:53.525833 6315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:53.527011 6315 hybrid_clock.cc:648] HybridClock initialized: now 1786346333526992 us; error 30 us; skew 500 ppm
Aug 10 07:18:53 dist-test-slave-10pc krb5kdc[6301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346333, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:53.529991 6315 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:18:53.530483 6315 webserver.cc:460] Webserver started at http://127.0.0.1:42085/ using document root <none> and password file <none>
I20260810 07:18:53.530750 6315 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:53.530798 6315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:53.530973 6315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:53.532107 6315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "69f988edfcd64948a2eaf371fb603290"
format_stamp: "Formatted at 2026-08-10 07:18:53 on dist-test-slave-10pc"
server_key: "d9554a6d44eb3943a3255cc8091b7db3"
server_key_iv: "fc5ecac70f0546f09f2c38ebfa8f78e8"
server_key_version: "encryptionkey@0"
I20260810 07:18:53.532567 6315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "69f988edfcd64948a2eaf371fb603290"
format_stamp: "Formatted at 2026-08-10 07:18:53 on dist-test-slave-10pc"
server_key: "d9554a6d44eb3943a3255cc8091b7db3"
server_key_iv: "fc5ecac70f0546f09f2c38ebfa8f78e8"
server_key_version: "encryptionkey@0"
I20260810 07:18:53.534001 6315 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:18:53.534838 6331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:53.535032 6315 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 07:18:53.535130 6315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "69f988edfcd64948a2eaf371fb603290"
format_stamp: "Formatted at 2026-08-10 07:18:53 on dist-test-slave-10pc"
server_key: "d9554a6d44eb3943a3255cc8091b7db3"
server_key_iv: "fc5ecac70f0546f09f2c38ebfa8f78e8"
server_key_version: "encryptionkey@0"
I20260810 07:18:53.535207 6315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:53.560400 6315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:53.563024 6315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:53.563203 6315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:53.567566 6315 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46793
I20260810 07:18:53.567612 6387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46793 every 8 connection(s)
I20260810 07:18:53.568004 6315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:18:53.568949 6388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:53.571398 6388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69f988edfcd64948a2eaf371fb603290: Bootstrap starting.
I20260810 07:18:53.572038 6388 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69f988edfcd64948a2eaf371fb603290: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:53.572352 6388 log.cc:824] T 00000000000000000000000000000000 P 69f988edfcd64948a2eaf371fb603290: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:53.573172 6388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69f988edfcd64948a2eaf371fb603290: No bootstrap required, opened a new log
I20260810 07:18:53.574890 6388 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69f988edfcd64948a2eaf371fb603290 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69f988edfcd64948a2eaf371fb603290" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46793 } }
I20260810 07:18:53.575112 6388 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69f988edfcd64948a2eaf371fb603290 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:53.575157 6388 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69f988edfcd64948a2eaf371fb603290 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69f988edfcd64948a2eaf371fb603290, State: Initialized, Role: FOLLOWER
I20260810 07:18:53.575201 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 6315
I20260810 07:18:53.575323 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:18:53.575371 6388 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69f988edfcd64948a2eaf371fb603290 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69f988edfcd64948a2eaf371fb603290" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46793 } }
I20260810 07:18:53.575471 6388 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69f988edfcd64948a2eaf371fb603290 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:18:53.575503 6388 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69f988edfcd64948a2eaf371fb603290 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:18:53.575536 16229 external_mini_cluster.cc:1428] Setting key f37f60476ec11369890f76e223315799
I20260810 07:18:53.575572 6388 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69f988edfcd64948a2eaf371fb603290 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:53.576411 6388 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69f988edfcd64948a2eaf371fb603290 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69f988edfcd64948a2eaf371fb603290" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46793 } }
I20260810 07:18:53.576596 6388 leader_election.cc:302] T 00000000000000000000000000000000 P 69f988edfcd64948a2eaf371fb603290 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69f988edfcd64948a2eaf371fb603290; no voters:
I20260810 07:18:53.576778 6388 leader_election.cc:288] T 00000000000000000000000000000000 P 69f988edfcd64948a2eaf371fb603290 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:18:53.576819 6393 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69f988edfcd64948a2eaf371fb603290 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:18:53.576977 6393 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69f988edfcd64948a2eaf371fb603290 [term 1 LEADER]: Becoming Leader. State: Replica: 69f988edfcd64948a2eaf371fb603290, State: Running, Role: LEADER
I20260810 07:18:53.577080 6388 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69f988edfcd64948a2eaf371fb603290 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:18:53.577160 6393 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69f988edfcd64948a2eaf371fb603290 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69f988edfcd64948a2eaf371fb603290" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46793 } }
I20260810 07:18:53.577564 6393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69f988edfcd64948a2eaf371fb603290 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69f988edfcd64948a2eaf371fb603290. Latest consensus state: current_term: 1 leader_uuid: "69f988edfcd64948a2eaf371fb603290" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69f988edfcd64948a2eaf371fb603290" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46793 } } }
I20260810 07:18:53.577731 6393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69f988edfcd64948a2eaf371fb603290 [sys.catalog]: This master's current role is: LEADER
I20260810 07:18:53.578006 6404 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:18:53.578168 6394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69f988edfcd64948a2eaf371fb603290 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69f988edfcd64948a2eaf371fb603290" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69f988edfcd64948a2eaf371fb603290" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46793 } } }
I20260810 07:18:53.578341 6394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69f988edfcd64948a2eaf371fb603290 [sys.catalog]: This master's current role is: LEADER
I20260810 07:18:53.578601 6404 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:18:53.580507 6404 catalog_manager.cc:1347] Generated new cluster ID: 42bbb993b2ec4f7885385f5d7240f54e
I20260810 07:18:53.580567 6404 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 07:18:53 dist-test-slave-10pc krb5kdc[6301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 07:18:53.588791 6392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:53.576427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56023 (local address 127.0.0.1:46793)
0810 07:18:53.576661 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:53.576679 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:53.576735 (+ 56us) server_negotiation.cc:407] Connection header received
0810 07:18:53.576889 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:53.576898 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:53.576967 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:53.577122 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:53.577936 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:53.579224 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:53.580294 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:53.580920 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:53.581030 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:53.583919 (+ 2889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:53.583924 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:53.583924 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:53.583938 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:53.586690 (+ 2752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:53.587113 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:53.587117 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:53.587117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:53.587223 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:53.587394 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:53.587394 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:53.587394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:53.587720 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:53.587846 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:53.588361 (+ 515us) server_negotiation.cc:299] Negotiation successful
0810 07:18:53.588532 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":147,"threads_started":1}
I20260810 07:18:53.589895 16229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:18:53.599023 6392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:53.591487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57743 (local address 127.0.0.1:46793)
0810 07:18:53.591524 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:53.591526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:53.591675 (+ 149us) server_negotiation.cc:407] Connection header received
0810 07:18:53.591735 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:53.591736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:53.591748 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:53.591799 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:53.592496 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:53.593239 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:53.594302 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:53.594621 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:53.594686 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:53.596178 (+ 1492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:53.596179 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:53.596180 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:53.596187 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:53.597815 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:53.598229 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:53.598230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:53.598230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:53.598289 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:53.598498 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:53.598498 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:53.598499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:53.598620 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:53.598709 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:53.598846 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 07:18:53.598921 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 07:18:53.607090 6404 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:18:53.607707 6404 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:18:53.614104 6404 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69f988edfcd64948a2eaf371fb603290: Generated new TSK 0
I20260810 07:18:53.614333 6404 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:18:53.621236 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 6315
2026-08-10T07:18:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1439 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:18:53 dist-test-slave-10pc krb5kdc[6421](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:18:53 dist-test-slave-10pc krb5kdc[6421](info): set up 2 sockets
Aug 10 07:18:53 dist-test-slave-10pc krb5kdc[6421](info): commencing operation
krb5kdc: starting...
W20260810 07:18:54.711793 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:18:54 dist-test-slave-10pc krb5kdc[6421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:18:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:18:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:18:54.853405 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43163
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43163
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:18:54.942611 6437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:54.942910 6437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:54.942957 6437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:54.944347 6437 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:18:54.944419 6437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:54.944444 6437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:54.944502 6437 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:18:54.944535 6437 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:18:54.946316 6437 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43163
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:54.946650 6437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:54.946913 6437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:54.952621 6443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:54.952665 6442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:54.952864 6437 server_base.cc:1034] running on GCE node
W20260810 07:18:54.952621 6445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:54.953224 6437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:54.953567 6437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:54.954764 6437 hybrid_clock.cc:648] HybridClock initialized: now 1786346334954674 us; error 105 us; skew 500 ppm
Aug 10 07:18:54 dist-test-slave-10pc krb5kdc[6421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346334, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:54.968044 6437 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:18:54.968748 6437 webserver.cc:460] Webserver started at http://127.0.0.1:38333/ using document root <none> and password file <none>
I20260810 07:18:54.969014 6437 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:54.969146 6437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:54.969331 6437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:54.970548 6437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "4f184d98a085476b90aefb223cf0f07a"
format_stamp: "Formatted at 2026-08-10 07:18:54 on dist-test-slave-10pc"
server_key: "a905b2a7a7e5e643236cec60a069021e"
server_key_iv: "f80ea1caa678da7d0303e962c8b80629"
server_key_version: "encryptionkey@0"
I20260810 07:18:54.970963 6437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "4f184d98a085476b90aefb223cf0f07a"
format_stamp: "Formatted at 2026-08-10 07:18:54 on dist-test-slave-10pc"
server_key: "a905b2a7a7e5e643236cec60a069021e"
server_key_iv: "f80ea1caa678da7d0303e962c8b80629"
server_key_version: "encryptionkey@0"
I20260810 07:18:54.972596 6437 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:18:54.973536 6453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:54.973752 6437 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:18:54.973867 6437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "4f184d98a085476b90aefb223cf0f07a"
format_stamp: "Formatted at 2026-08-10 07:18:54 on dist-test-slave-10pc"
server_key: "a905b2a7a7e5e643236cec60a069021e"
server_key_iv: "f80ea1caa678da7d0303e962c8b80629"
server_key_version: "encryptionkey@0"
I20260810 07:18:54.973955 6437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:54.992336 6437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:54.994892 6437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:54.995066 6437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:54.999816 6437 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43163
I20260810 07:18:54.999854 6511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43163 every 8 connection(s)
I20260810 07:18:55.000264 6437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:18:55.000644 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 6437
I20260810 07:18:55.000797 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:18:55.001102 16229 external_mini_cluster.cc:1428] Setting key 832f988d8dcfcc690946c64a8a432834
I20260810 07:18:55.001516 6512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:55.005394 6512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f184d98a085476b90aefb223cf0f07a: Bootstrap starting.
I20260810 07:18:55.006250 6512 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f184d98a085476b90aefb223cf0f07a: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:55.006592 6512 log.cc:824] T 00000000000000000000000000000000 P 4f184d98a085476b90aefb223cf0f07a: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:55.007455 6512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f184d98a085476b90aefb223cf0f07a: No bootstrap required, opened a new log
I20260810 07:18:55.007494 6515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:55.001719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50027 (local address 127.0.0.1:43163)
0810 07:18:55.001928 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:55.001937 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:55.001965 (+ 28us) server_negotiation.cc:407] Connection header received
0810 07:18:55.002438 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:55.002445 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:55.002498 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:55.002631 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:55.002912 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:55.003832 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:55.005149 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:55.005717 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:55.005818 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:55.006759 (+ 941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:55.006763 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:55.006764 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:18:55.006957 (+ 193us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:55.006999 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:18:55.007016 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:55.007113 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:55.007200 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 07:18:55.007282 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":109,"threads_started":1}
I20260810 07:18:55.008764 16229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:18:55.009876 6512 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f184d98a085476b90aefb223cf0f07a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f184d98a085476b90aefb223cf0f07a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43163 } }
I20260810 07:18:55.010097 6512 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f184d98a085476b90aefb223cf0f07a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:55.010146 6512 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f184d98a085476b90aefb223cf0f07a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f184d98a085476b90aefb223cf0f07a, State: Initialized, Role: FOLLOWER
I20260810 07:18:55.010283 6512 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f184d98a085476b90aefb223cf0f07a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f184d98a085476b90aefb223cf0f07a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43163 } }
I20260810 07:18:55.010390 6512 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f184d98a085476b90aefb223cf0f07a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:18:55.010483 6512 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f184d98a085476b90aefb223cf0f07a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:18:55.010550 6512 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f184d98a085476b90aefb223cf0f07a [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:55.011401 6512 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f184d98a085476b90aefb223cf0f07a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f184d98a085476b90aefb223cf0f07a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43163 } }
I20260810 07:18:55.011524 6512 leader_election.cc:302] T 00000000000000000000000000000000 P 4f184d98a085476b90aefb223cf0f07a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f184d98a085476b90aefb223cf0f07a; no voters:
I20260810 07:18:55.011716 6512 leader_election.cc:288] T 00000000000000000000000000000000 P 4f184d98a085476b90aefb223cf0f07a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:18:55.011765 6522 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f184d98a085476b90aefb223cf0f07a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:18:55.011952 6522 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f184d98a085476b90aefb223cf0f07a [term 1 LEADER]: Becoming Leader. State: Replica: 4f184d98a085476b90aefb223cf0f07a, State: Running, Role: LEADER
I20260810 07:18:55.012049 6512 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f184d98a085476b90aefb223cf0f07a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:18:55.012147 6522 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f184d98a085476b90aefb223cf0f07a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f184d98a085476b90aefb223cf0f07a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43163 } }
I20260810 07:18:55.012635 6522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f184d98a085476b90aefb223cf0f07a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f184d98a085476b90aefb223cf0f07a. Latest consensus state: current_term: 1 leader_uuid: "4f184d98a085476b90aefb223cf0f07a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f184d98a085476b90aefb223cf0f07a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43163 } } }
I20260810 07:18:55.012770 6522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f184d98a085476b90aefb223cf0f07a [sys.catalog]: This master's current role is: LEADER
I20260810 07:18:55.013079 6531 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:18:55.013571 6524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f184d98a085476b90aefb223cf0f07a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f184d98a085476b90aefb223cf0f07a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f184d98a085476b90aefb223cf0f07a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43163 } } }
I20260810 07:18:55.013741 6524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f184d98a085476b90aefb223cf0f07a [sys.catalog]: This master's current role is: LEADER
I20260810 07:18:55.013774 6531 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:18:55.014045 6515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:55.010333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46661 (local address 127.0.0.1:43163)
0810 07:18:55.010377 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:55.010379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:55.010414 (+ 35us) server_negotiation.cc:407] Connection header received
0810 07:18:55.010483 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:55.010484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:55.010496 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:55.010571 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:55.010769 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:55.011463 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:55.012538 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:55.012858 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:55.012926 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:55.013679 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:55.013680 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:55.013680 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:18:55.013685 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:55.013703 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:18:55.013708 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:55.013750 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:55.013940 (+ 190us) server_negotiation.cc:299] Negotiation successful
0810 07:18:55.013969 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 07:18:55.015633 6531 catalog_manager.cc:1347] Generated new cluster ID: dbb18ae154814905b924e3a712dff880
I20260810 07:18:55.015691 6531 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:18:55.021981 6531 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:18:55.022579 6531 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:18:55.029567 6531 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f184d98a085476b90aefb223cf0f07a: Generated new TSK 0
I20260810 07:18:55.029803 6531 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:18:55.032627 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 6437
2026-08-10T07:18:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1412 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:18:55 dist-test-slave-10pc krb5kdc[6544](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:18:55 dist-test-slave-10pc krb5kdc[6544](info): set up 2 sockets
Aug 10 07:18:55 dist-test-slave-10pc krb5kdc[6544](info): commencing operation
krb5kdc: starting...
W20260810 07:18:56.148134 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:18:56 dist-test-slave-10pc krb5kdc[6544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:18:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:18:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:18:56.288345 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33307
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:18:56.377643 6559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:56.377887 6559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:56.377911 6559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:56.379127 6559 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:18:56.379172 6559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:56.379189 6559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:56.379201 6559 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:18:56.379213 6559 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:18:56.380800 6559 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:56.381036 6559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:56.381244 6559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:56.386945 6565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:56.387053 6564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:56.386922 6567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:56.387096 6559 server_base.cc:1034] running on GCE node
I20260810 07:18:56.387493 6559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:56.387787 6559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:56.388976 6559 hybrid_clock.cc:648] HybridClock initialized: now 1786346336388942 us; error 64 us; skew 500 ppm
Aug 10 07:18:56 dist-test-slave-10pc krb5kdc[6544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346336, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:56.391754 6559 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:18:56.392271 6559 webserver.cc:460] Webserver started at http://127.0.0.1:41069/ using document root <none> and password file <none>
I20260810 07:18:56.392573 6559 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:56.392654 6559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:56.392820 6559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:56.393944 6559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "9fd96657f2c044f5b0d26ef1088fc9f0"
format_stamp: "Formatted at 2026-08-10 07:18:56 on dist-test-slave-10pc"
server_key: "d8ca34a38b83abf89dceb07fe1d3786d"
server_key_iv: "d455e167237bfcff599a982cdb7e2d8e"
server_key_version: "encryptionkey@0"
I20260810 07:18:56.394330 6559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "9fd96657f2c044f5b0d26ef1088fc9f0"
format_stamp: "Formatted at 2026-08-10 07:18:56 on dist-test-slave-10pc"
server_key: "d8ca34a38b83abf89dceb07fe1d3786d"
server_key_iv: "d455e167237bfcff599a982cdb7e2d8e"
server_key_version: "encryptionkey@0"
I20260810 07:18:56.395782 6559 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:18:56.396539 6574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:56.396749 6559 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:18:56.396849 6559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "9fd96657f2c044f5b0d26ef1088fc9f0"
format_stamp: "Formatted at 2026-08-10 07:18:56 on dist-test-slave-10pc"
server_key: "d8ca34a38b83abf89dceb07fe1d3786d"
server_key_iv: "d455e167237bfcff599a982cdb7e2d8e"
server_key_version: "encryptionkey@0"
I20260810 07:18:56.396925 6559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:56.400584 6559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:56.402989 6559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:56.403149 6559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:56.407467 6559 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33307
I20260810 07:18:56.407523 6631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33307 every 8 connection(s)
I20260810 07:18:56.407843 6559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:18:56.408807 6632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:56.411157 6632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fd96657f2c044f5b0d26ef1088fc9f0: Bootstrap starting.
I20260810 07:18:56.411713 6632 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9fd96657f2c044f5b0d26ef1088fc9f0: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:56.411971 6632 log.cc:824] T 00000000000000000000000000000000 P 9fd96657f2c044f5b0d26ef1088fc9f0: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:56.412725 6632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fd96657f2c044f5b0d26ef1088fc9f0: No bootstrap required, opened a new log
I20260810 07:18:56.414379 6632 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fd96657f2c044f5b0d26ef1088fc9f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fd96657f2c044f5b0d26ef1088fc9f0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33307 } }
I20260810 07:18:56.414536 6632 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9fd96657f2c044f5b0d26ef1088fc9f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:56.414569 6632 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fd96657f2c044f5b0d26ef1088fc9f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fd96657f2c044f5b0d26ef1088fc9f0, State: Initialized, Role: FOLLOWER
I20260810 07:18:56.414740 6632 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fd96657f2c044f5b0d26ef1088fc9f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fd96657f2c044f5b0d26ef1088fc9f0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33307 } }
I20260810 07:18:56.414844 6632 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9fd96657f2c044f5b0d26ef1088fc9f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:18:56.414848 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 6559
I20260810 07:18:56.414880 6632 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9fd96657f2c044f5b0d26ef1088fc9f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:18:56.414932 6632 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fd96657f2c044f5b0d26ef1088fc9f0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:56.414980 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:18:56.415190 16229 external_mini_cluster.cc:1428] Setting key f2e01e89a1a981d2b7e49a55cbf95247
I20260810 07:18:56.415711 6632 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9fd96657f2c044f5b0d26ef1088fc9f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fd96657f2c044f5b0d26ef1088fc9f0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33307 } }
I20260810 07:18:56.415840 6632 leader_election.cc:302] T 00000000000000000000000000000000 P 9fd96657f2c044f5b0d26ef1088fc9f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9fd96657f2c044f5b0d26ef1088fc9f0; no voters:
I20260810 07:18:56.416021 6632 leader_election.cc:288] T 00000000000000000000000000000000 P 9fd96657f2c044f5b0d26ef1088fc9f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:18:56.416050 6638 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9fd96657f2c044f5b0d26ef1088fc9f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:18:56.416157 6638 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9fd96657f2c044f5b0d26ef1088fc9f0 [term 1 LEADER]: Becoming Leader. State: Replica: 9fd96657f2c044f5b0d26ef1088fc9f0, State: Running, Role: LEADER
I20260810 07:18:56.416252 6638 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9fd96657f2c044f5b0d26ef1088fc9f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fd96657f2c044f5b0d26ef1088fc9f0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33307 } }
I20260810 07:18:56.416303 6632 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fd96657f2c044f5b0d26ef1088fc9f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:18:56.416643 6638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fd96657f2c044f5b0d26ef1088fc9f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9fd96657f2c044f5b0d26ef1088fc9f0. Latest consensus state: current_term: 1 leader_uuid: "9fd96657f2c044f5b0d26ef1088fc9f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fd96657f2c044f5b0d26ef1088fc9f0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33307 } } }
I20260810 07:18:56.416733 6638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fd96657f2c044f5b0d26ef1088fc9f0 [sys.catalog]: This master's current role is: LEADER
I20260810 07:18:56.416695 6639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fd96657f2c044f5b0d26ef1088fc9f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9fd96657f2c044f5b0d26ef1088fc9f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fd96657f2c044f5b0d26ef1088fc9f0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33307 } } }
I20260810 07:18:56.416941 6639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fd96657f2c044f5b0d26ef1088fc9f0 [sys.catalog]: This master's current role is: LEADER
I20260810 07:18:56.416961 6642 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:18:56.417595 6642 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:18:56.418269 6637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:56.415722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37771 (local address 127.0.0.1:33307)
0810 07:18:56.415910 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:56.415920 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:56.415955 (+ 35us) server_negotiation.cc:407] Connection header received
0810 07:18:56.416093 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:56.416102 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:56.416134 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:56.416585 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:56.417593 (+ 1008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:56.417597 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:56.417597 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:18:56.417602 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:56.417658 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:18:56.417675 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:56.417740 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:56.418035 (+ 295us) server_negotiation.cc:299] Negotiation successful
0810 07:18:56.418080 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":76,"threads_started":1}
I20260810 07:18:56.419270 16229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:18:56.419526 6642 catalog_manager.cc:1347] Generated new cluster ID: 4e6a087379eb49428a2213a6c6d76790
I20260810 07:18:56.419574 6642 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:18:56.429198 6642 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:18:56.429440 6642 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:18:56.429581 6637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:56.420878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41939 (local address 127.0.0.1:33307)
0810 07:18:56.420962 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:56.420963 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:56.421188 (+ 225us) server_negotiation.cc:407] Connection header received
0810 07:18:56.421217 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:56.421218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:56.421224 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:56.421290 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:56.429326 (+ 8036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:56.429328 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:56.429329 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:18:56.429335 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:56.429355 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:18:56.429360 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:56.429434 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:56.429500 (+ 66us) server_negotiation.cc:299] Negotiation successful
0810 07:18:56.429517 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 07:18:56.432354 6642 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9fd96657f2c044f5b0d26ef1088fc9f0: Generated new TSK 0
I20260810 07:18:56.432618 6642 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:18:56.437953 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 6559
2026-08-10T07:18:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1403 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:18:56 dist-test-slave-10pc krb5kdc[6665](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:18:56 dist-test-slave-10pc krb5kdc[6665](info): set up 2 sockets
Aug 10 07:18:56 dist-test-slave-10pc krb5kdc[6665](info): commencing operation
krb5kdc: starting...
W20260810 07:18:57.552351 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:18:57 dist-test-slave-10pc krb5kdc[6665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:18:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:18:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:18:57.693590 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46825
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46825
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:18:57.786195 6678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:57.786465 6678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:57.786494 6678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:57.787837 6678 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:18:57.787897 6678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:57.787912 6678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:57.787926 6678 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:18:57.787938 6678 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:18:57.789865 6678 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:57.790180 6678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:57.790452 6678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:57.796243 6683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:57.796223 6684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:57.796272 6678 server_base.cc:1034] running on GCE node
W20260810 07:18:57.796223 6686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:57.796833 6678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:57.797170 6678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:57.798352 6678 hybrid_clock.cc:648] HybridClock initialized: now 1786346337798341 us; error 35 us; skew 500 ppm
Aug 10 07:18:57 dist-test-slave-10pc krb5kdc[6665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346337, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:57.801134 6678 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:18:57.801637 6678 webserver.cc:460] Webserver started at http://127.0.0.1:37577/ using document root <none> and password file <none>
I20260810 07:18:57.801883 6678 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:57.801934 6678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:57.802074 6678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:57.803406 6678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "00b2f22270474afeb1810d9aef63bde0"
format_stamp: "Formatted at 2026-08-10 07:18:57 on dist-test-slave-10pc"
server_key: "9eeee940624ca0121d1ccd0a884588c9"
server_key_iv: "d50e7455fbf2ce2db59ae54ac5d8e95a"
server_key_version: "encryptionkey@0"
I20260810 07:18:57.803802 6678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "00b2f22270474afeb1810d9aef63bde0"
format_stamp: "Formatted at 2026-08-10 07:18:57 on dist-test-slave-10pc"
server_key: "9eeee940624ca0121d1ccd0a884588c9"
server_key_iv: "d50e7455fbf2ce2db59ae54ac5d8e95a"
server_key_version: "encryptionkey@0"
I20260810 07:18:57.805431 6678 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:18:57.806268 6694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:57.806466 6678 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:18:57.806565 6678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "00b2f22270474afeb1810d9aef63bde0"
format_stamp: "Formatted at 2026-08-10 07:18:57 on dist-test-slave-10pc"
server_key: "9eeee940624ca0121d1ccd0a884588c9"
server_key_iv: "d50e7455fbf2ce2db59ae54ac5d8e95a"
server_key_version: "encryptionkey@0"
I20260810 07:18:57.806648 6678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:57.816247 6678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:57.818497 6678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:57.818675 6678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:57.822988 6678 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46825
I20260810 07:18:57.823037 6750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46825 every 8 connection(s)
I20260810 07:18:57.823436 6678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:18:57.824417 6752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:57.826997 6752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00b2f22270474afeb1810d9aef63bde0: Bootstrap starting.
I20260810 07:18:57.827662 6752 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00b2f22270474afeb1810d9aef63bde0: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:57.827997 6752 log.cc:824] T 00000000000000000000000000000000 P 00b2f22270474afeb1810d9aef63bde0: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:57.828814 6752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00b2f22270474afeb1810d9aef63bde0: No bootstrap required, opened a new log
I20260810 07:18:57.830515 6752 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00b2f22270474afeb1810d9aef63bde0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b2f22270474afeb1810d9aef63bde0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46825 } }
I20260810 07:18:57.830704 6752 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00b2f22270474afeb1810d9aef63bde0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:57.830729 6752 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00b2f22270474afeb1810d9aef63bde0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00b2f22270474afeb1810d9aef63bde0, State: Initialized, Role: FOLLOWER
I20260810 07:18:57.830860 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 6678
I20260810 07:18:57.830986 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:18:57.830930 6752 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00b2f22270474afeb1810d9aef63bde0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b2f22270474afeb1810d9aef63bde0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46825 } }
I20260810 07:18:57.831037 6752 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00b2f22270474afeb1810d9aef63bde0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:18:57.831094 6752 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00b2f22270474afeb1810d9aef63bde0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:18:57.831146 6752 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00b2f22270474afeb1810d9aef63bde0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:57.831228 16229 external_mini_cluster.cc:1428] Setting key b4c4c36a48668a383736e720a26fa2e3
I20260810 07:18:57.831915 6752 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00b2f22270474afeb1810d9aef63bde0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b2f22270474afeb1810d9aef63bde0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46825 } }
I20260810 07:18:57.832051 6752 leader_election.cc:302] T 00000000000000000000000000000000 P 00b2f22270474afeb1810d9aef63bde0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00b2f22270474afeb1810d9aef63bde0; no voters:
I20260810 07:18:57.832253 6752 leader_election.cc:288] T 00000000000000000000000000000000 P 00b2f22270474afeb1810d9aef63bde0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:18:57.832283 6757 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00b2f22270474afeb1810d9aef63bde0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:18:57.832393 6757 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00b2f22270474afeb1810d9aef63bde0 [term 1 LEADER]: Becoming Leader. State: Replica: 00b2f22270474afeb1810d9aef63bde0, State: Running, Role: LEADER
I20260810 07:18:57.832561 6757 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00b2f22270474afeb1810d9aef63bde0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b2f22270474afeb1810d9aef63bde0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46825 } }
I20260810 07:18:57.832566 6752 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00b2f22270474afeb1810d9aef63bde0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:18:57.833312 6758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00b2f22270474afeb1810d9aef63bde0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00b2f22270474afeb1810d9aef63bde0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b2f22270474afeb1810d9aef63bde0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46825 } } }
I20260810 07:18:57.833421 6758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00b2f22270474afeb1810d9aef63bde0 [sys.catalog]: This master's current role is: LEADER
I20260810 07:18:57.833648 6759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00b2f22270474afeb1810d9aef63bde0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00b2f22270474afeb1810d9aef63bde0. Latest consensus state: current_term: 1 leader_uuid: "00b2f22270474afeb1810d9aef63bde0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b2f22270474afeb1810d9aef63bde0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46825 } } }
I20260810 07:18:57.833748 6759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00b2f22270474afeb1810d9aef63bde0 [sys.catalog]: This master's current role is: LEADER
I20260810 07:18:57.833791 6761 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:18:57.834221 6761 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:18:57.836545 6761 catalog_manager.cc:1347] Generated new cluster ID: f9d0289a2aab40048ef987d98ffff413
I20260810 07:18:57.836607 6761 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 07:18:57 dist-test-slave-10pc krb5kdc[6665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 07:18:57.842633 6756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:57.831803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.141:36521 (local address 127.0.0.1:46825)
0810 07:18:57.832013 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:57.832024 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:57.832062 (+ 38us) server_negotiation.cc:407] Connection header received
0810 07:18:57.832201 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:57.832207 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:57.832256 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:57.833161 (+ 905us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:57.833287 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:57.834626 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:57.835750 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:57.836393 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:57.836924 (+ 531us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:57.839036 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:57.839040 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:57.839041 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:57.839053 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:57.840922 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:57.841336 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:57.841339 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:57.841339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:57.841439 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:57.841680 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:57.841681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:57.841681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:57.841933 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:57.842133 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:57.842252 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 07:18:57.842471 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":106,"threads_started":1}
I20260810 07:18:57.843636 16229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:18:57.849324 6761 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:18:57.850404 6761 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:18:57.851524 6756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:57.845111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.141:34637 (local address 127.0.0.1:46825)
0810 07:18:57.845167 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:57.845170 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:57.845244 (+ 74us) server_negotiation.cc:407] Connection header received
0810 07:18:57.845332 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:57.845332 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:57.845342 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:57.845386 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:57.845905 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:57.846402 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:57.847109 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:57.847356 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:57.847429 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:57.848750 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:57.848752 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:57.848752 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:57.848759 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:57.850321 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:57.850719 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:57.850720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:57.850720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:57.850783 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:57.851003 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:57.851004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:57.851005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:57.851119 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:57.851231 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:57.851360 (+ 129us) server_negotiation.cc:299] Negotiation successful
0810 07:18:57.851427 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 07:18:57.856986 6761 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00b2f22270474afeb1810d9aef63bde0: Generated new TSK 0
I20260810 07:18:57.857200 6761 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:18:57.863104 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 6678
W20260810 07:18:57.871454 6663 connection.cc:535] client connection to 127.0.0.1:46825 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46825): Connection reset by peer (error 104)
2026-08-10T07:18:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1424 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:18:57 dist-test-slave-10pc krb5kdc[6786](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:18:57 dist-test-slave-10pc krb5kdc[6786](info): set up 2 sockets
Aug 10 07:18:57 dist-test-slave-10pc krb5kdc[6786](info): commencing operation
krb5kdc: starting...
W20260810 07:18:58.962589 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:18:59 dist-test-slave-10pc krb5kdc[6786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:18:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:18:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:18:59.103474 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39187
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39187
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:18:59.191273 6801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:59.191540 6801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:59.191582 6801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:59.192853 6801 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:18:59.192915 6801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:59.192941 6801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:59.192974 6801 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:18:59.193002 6801 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:18:59.194518 6801 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39187
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:18:59.194861 6801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:59.195124 6801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:59.200232 6807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:59.200376 6801 server_base.cc:1034] running on GCE node
W20260810 07:18:59.200513 6806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:59.200531 6809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:59.200841 6801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:59.201187 6801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:59.202392 6801 hybrid_clock.cc:648] HybridClock initialized: now 1786346339202379 us; error 34 us; skew 500 ppm
Aug 10 07:18:59 dist-test-slave-10pc krb5kdc[6786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346339, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:59.205196 6801 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:18:59.205678 6801 webserver.cc:460] Webserver started at http://127.0.0.1:38947/ using document root <none> and password file <none>
I20260810 07:18:59.205971 6801 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:59.206097 6801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:59.206247 6801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:59.207360 6801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "524b458cbeff4bba9d52743d82b3d8c8"
format_stamp: "Formatted at 2026-08-10 07:18:59 on dist-test-slave-10pc"
server_key: "21cc75c5c4760aa4b1a14fdf99a94bb4"
server_key_iv: "17430082691f7f5f595013b6fdeb529a"
server_key_version: "encryptionkey@0"
I20260810 07:18:59.207712 6801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "524b458cbeff4bba9d52743d82b3d8c8"
format_stamp: "Formatted at 2026-08-10 07:18:59 on dist-test-slave-10pc"
server_key: "21cc75c5c4760aa4b1a14fdf99a94bb4"
server_key_iv: "17430082691f7f5f595013b6fdeb529a"
server_key_version: "encryptionkey@0"
I20260810 07:18:59.209182 6801 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:18:59.209949 6817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:59.210137 6801 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 07:18:59.210232 6801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "524b458cbeff4bba9d52743d82b3d8c8"
format_stamp: "Formatted at 2026-08-10 07:18:59 on dist-test-slave-10pc"
server_key: "21cc75c5c4760aa4b1a14fdf99a94bb4"
server_key_iv: "17430082691f7f5f595013b6fdeb529a"
server_key_version: "encryptionkey@0"
I20260810 07:18:59.210317 6801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:59.234288 6801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:59.236794 6801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:59.237004 6801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:59.241348 6801 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39187
I20260810 07:18:59.241400 6875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39187 every 8 connection(s)
I20260810 07:18:59.241829 6801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:18:59.242853 6876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:59.245469 6876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 524b458cbeff4bba9d52743d82b3d8c8: Bootstrap starting.
I20260810 07:18:59.246124 6876 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 524b458cbeff4bba9d52743d82b3d8c8: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:59.246436 6876 log.cc:824] T 00000000000000000000000000000000 P 524b458cbeff4bba9d52743d82b3d8c8: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:59.247228 6876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 524b458cbeff4bba9d52743d82b3d8c8: No bootstrap required, opened a new log
I20260810 07:18:59.248960 6876 raft_consensus.cc:348] T 00000000000000000000000000000000 P 524b458cbeff4bba9d52743d82b3d8c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "524b458cbeff4bba9d52743d82b3d8c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39187 } }
I20260810 07:18:59.249114 6876 raft_consensus.cc:374] T 00000000000000000000000000000000 P 524b458cbeff4bba9d52743d82b3d8c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:59.249136 6876 raft_consensus.cc:729] T 00000000000000000000000000000000 P 524b458cbeff4bba9d52743d82b3d8c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 524b458cbeff4bba9d52743d82b3d8c8, State: Initialized, Role: FOLLOWER
I20260810 07:18:59.249336 6876 consensus_queue.cc:260] T 00000000000000000000000000000000 P 524b458cbeff4bba9d52743d82b3d8c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "524b458cbeff4bba9d52743d82b3d8c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39187 } }
I20260810 07:18:59.249428 6876 raft_consensus.cc:388] T 00000000000000000000000000000000 P 524b458cbeff4bba9d52743d82b3d8c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:18:59.249480 6876 raft_consensus.cc:482] T 00000000000000000000000000000000 P 524b458cbeff4bba9d52743d82b3d8c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:18:59.249536 6876 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 524b458cbeff4bba9d52743d82b3d8c8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:59.250114 6876 raft_consensus.cc:504] T 00000000000000000000000000000000 P 524b458cbeff4bba9d52743d82b3d8c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "524b458cbeff4bba9d52743d82b3d8c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39187 } }
I20260810 07:18:59.250264 6876 leader_election.cc:302] T 00000000000000000000000000000000 P 524b458cbeff4bba9d52743d82b3d8c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 524b458cbeff4bba9d52743d82b3d8c8; no voters:
I20260810 07:18:59.250382 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 6801
I20260810 07:18:59.250461 6876 leader_election.cc:288] T 00000000000000000000000000000000 P 524b458cbeff4bba9d52743d82b3d8c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:18:59.250524 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:18:59.250514 6879 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 524b458cbeff4bba9d52743d82b3d8c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:18:59.250692 6879 raft_consensus.cc:686] T 00000000000000000000000000000000 P 524b458cbeff4bba9d52743d82b3d8c8 [term 1 LEADER]: Becoming Leader. State: Replica: 524b458cbeff4bba9d52743d82b3d8c8, State: Running, Role: LEADER
I20260810 07:18:59.250777 16229 external_mini_cluster.cc:1428] Setting key 0be65fefee5c208e9b8b65f5b383619e
I20260810 07:18:59.250808 6876 sys_catalog.cc:564] T 00000000000000000000000000000000 P 524b458cbeff4bba9d52743d82b3d8c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:18:59.250825 6879 consensus_queue.cc:237] T 00000000000000000000000000000000 P 524b458cbeff4bba9d52743d82b3d8c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "524b458cbeff4bba9d52743d82b3d8c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39187 } }
I20260810 07:18:59.251706 6880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 524b458cbeff4bba9d52743d82b3d8c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "524b458cbeff4bba9d52743d82b3d8c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "524b458cbeff4bba9d52743d82b3d8c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39187 } } }
I20260810 07:18:59.251819 6880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 524b458cbeff4bba9d52743d82b3d8c8 [sys.catalog]: This master's current role is: LEADER
I20260810 07:18:59.252142 6893 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:18:59.252547 6881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 524b458cbeff4bba9d52743d82b3d8c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 524b458cbeff4bba9d52743d82b3d8c8. Latest consensus state: current_term: 1 leader_uuid: "524b458cbeff4bba9d52743d82b3d8c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "524b458cbeff4bba9d52743d82b3d8c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39187 } } }
I20260810 07:18:59.252701 6881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 524b458cbeff4bba9d52743d82b3d8c8 [sys.catalog]: This master's current role is: LEADER
I20260810 07:18:59.252872 6893 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:18:59.254434 6893 catalog_manager.cc:1347] Generated new cluster ID: 0d41cc976b1f4f3f8b1ca40ce0f881cd
I20260810 07:18:59.254491 6893 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:18:59.258652 6888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:59.251381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.141:60959 (local address 127.0.0.1:39187)
0810 07:18:59.251601 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:59.251611 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:59.252123 (+ 512us) server_negotiation.cc:407] Connection header received
0810 07:18:59.252376 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:59.252384 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:59.252543 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:59.252773 (+ 230us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:59.253157 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:59.254687 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:59.255870 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:59.256556 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:59.256681 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:59.257692 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:59.257697 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:59.257697 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:18:59.257922 (+ 225us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:59.257965 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:18:59.257985 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:59.258071 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:59.258366 (+ 295us) server_negotiation.cc:299] Negotiation successful
0810 07:18:59.258400 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":96,"threads_started":1}
I20260810 07:18:59.259891 16229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:18:59.265264 6888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:59.261518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.141:45799 (local address 127.0.0.1:39187)
0810 07:18:59.261557 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:59.261559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:59.261730 (+ 171us) server_negotiation.cc:407] Connection header received
0810 07:18:59.261781 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:59.261782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:59.261793 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:59.261873 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:59.262004 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:59.262775 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:59.263781 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:59.264106 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:59.264220 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:59.264986 (+ 766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:59.264987 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:59.264988 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:18:59.264993 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:59.265014 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:18:59.265019 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:59.265088 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:59.265156 (+ 68us) server_negotiation.cc:299] Negotiation successful
0810 07:18:59.265171 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 07:18:59.269639 6893 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:18:59.270342 6893 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:18:59.280153 6893 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 524b458cbeff4bba9d52743d82b3d8c8: Generated new TSK 0
I20260810 07:18:59.280403 6893 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:18:59.284880 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 6801
W20260810 07:18:59.292732 6784 connection.cc:535] client connection to 127.0.0.1:39187 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39187): Connection reset by peer (error 104)
2026-08-10T07:18:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1421 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:18:59 dist-test-slave-10pc krb5kdc[6911](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:18:59 dist-test-slave-10pc krb5kdc[6911](info): set up 2 sockets
Aug 10 07:18:59 dist-test-slave-10pc krb5kdc[6911](info): commencing operation
krb5kdc: starting...
W20260810 07:19:00.387387 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:19:00 dist-test-slave-10pc krb5kdc[6911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:19:00.523671 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40977
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:00.611202 6926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:00.611459 6926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:00.611501 6926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:00.612830 6926 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:00.612900 6926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:00.612924 6926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:00.612947 6926 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:00.612985 6926 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:00.614539 6926 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40977
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:00.614832 6926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:00.615073 6926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:19:00.620378 6926 server_base.cc:1034] running on GCE node
W20260810 07:19:00.620321 6933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:00.620321 6932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:00.620347 6935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:00.620838 6926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:00.621169 6926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:00.622319 6926 hybrid_clock.cc:648] HybridClock initialized: now 1786346340622288 us; error 49 us; skew 500 ppm
Aug 10 07:19:00 dist-test-slave-10pc krb5kdc[6911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346340, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:00.630018 6926 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:19:00.630529 6926 webserver.cc:460] Webserver started at http://127.0.0.1:37169/ using document root <none> and password file <none>
I20260810 07:19:00.630784 6926 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:00.630854 6926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:00.631003 6926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:00.632179 6926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "cac0d233c47b415092c94f7c8b8d0bee"
format_stamp: "Formatted at 2026-08-10 07:19:00 on dist-test-slave-10pc"
server_key: "3ceefee57429f9709e7639790b5f4424"
server_key_iv: "6b64505fee51b8de8f81fc31788416a8"
server_key_version: "encryptionkey@0"
I20260810 07:19:00.632632 6926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "cac0d233c47b415092c94f7c8b8d0bee"
format_stamp: "Formatted at 2026-08-10 07:19:00 on dist-test-slave-10pc"
server_key: "3ceefee57429f9709e7639790b5f4424"
server_key_iv: "6b64505fee51b8de8f81fc31788416a8"
server_key_version: "encryptionkey@0"
I20260810 07:19:00.634106 6926 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260810 07:19:00.634941 6945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:00.635152 6926 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:00.635247 6926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "cac0d233c47b415092c94f7c8b8d0bee"
format_stamp: "Formatted at 2026-08-10 07:19:00 on dist-test-slave-10pc"
server_key: "3ceefee57429f9709e7639790b5f4424"
server_key_iv: "6b64505fee51b8de8f81fc31788416a8"
server_key_version: "encryptionkey@0"
I20260810 07:19:00.635329 6926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:00.638823 6926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:00.641346 6926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:00.641508 6926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:00.645604 6926 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40977
I20260810 07:19:00.645651 6999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40977 every 8 connection(s)
I20260810 07:19:00.646155 6926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:00.647051 7000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:00.649641 7000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cac0d233c47b415092c94f7c8b8d0bee: Bootstrap starting.
I20260810 07:19:00.649971 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 6926
I20260810 07:19:00.650141 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:19:00.650352 16229 external_mini_cluster.cc:1428] Setting key 16c4d4cf5e03d35ab45c135321756e0e
I20260810 07:19:00.650367 7000 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cac0d233c47b415092c94f7c8b8d0bee: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:00.650671 7000 log.cc:824] T 00000000000000000000000000000000 P cac0d233c47b415092c94f7c8b8d0bee: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:00.651510 7000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cac0d233c47b415092c94f7c8b8d0bee: No bootstrap required, opened a new log
I20260810 07:19:00.653069 7005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:00.650931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.141:41861 (local address 127.0.0.1:40977)
0810 07:19:00.651146 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:00.651157 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:00.651240 (+ 83us) server_negotiation.cc:407] Connection header received
0810 07:19:00.651729 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:00.651736 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:00.651764 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:00.651884 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:00.652602 (+ 718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:00.652606 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:00.652606 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:00.652610 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:00.652650 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:00.652661 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:00.652737 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:00.652807 (+ 70us) server_negotiation.cc:299] Negotiation successful
0810 07:19:00.652862 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":85,"threads_started":1}
I20260810 07:19:00.654124 16229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:19:00.654347 7000 raft_consensus.cc:348] T 00000000000000000000000000000000 P cac0d233c47b415092c94f7c8b8d0bee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cac0d233c47b415092c94f7c8b8d0bee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40977 } }
I20260810 07:19:00.654510 7000 raft_consensus.cc:374] T 00000000000000000000000000000000 P cac0d233c47b415092c94f7c8b8d0bee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:00.654568 7000 raft_consensus.cc:729] T 00000000000000000000000000000000 P cac0d233c47b415092c94f7c8b8d0bee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cac0d233c47b415092c94f7c8b8d0bee, State: Initialized, Role: FOLLOWER
I20260810 07:19:00.654685 7000 consensus_queue.cc:260] T 00000000000000000000000000000000 P cac0d233c47b415092c94f7c8b8d0bee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cac0d233c47b415092c94f7c8b8d0bee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40977 } }
I20260810 07:19:00.654767 7000 raft_consensus.cc:388] T 00000000000000000000000000000000 P cac0d233c47b415092c94f7c8b8d0bee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:00.654790 7000 raft_consensus.cc:482] T 00000000000000000000000000000000 P cac0d233c47b415092c94f7c8b8d0bee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:00.654853 7000 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cac0d233c47b415092c94f7c8b8d0bee [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:00.655450 7000 raft_consensus.cc:504] T 00000000000000000000000000000000 P cac0d233c47b415092c94f7c8b8d0bee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cac0d233c47b415092c94f7c8b8d0bee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40977 } }
I20260810 07:19:00.655579 7000 leader_election.cc:302] T 00000000000000000000000000000000 P cac0d233c47b415092c94f7c8b8d0bee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cac0d233c47b415092c94f7c8b8d0bee; no voters:
I20260810 07:19:00.655763 7000 leader_election.cc:288] T 00000000000000000000000000000000 P cac0d233c47b415092c94f7c8b8d0bee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:00.655781 7010 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cac0d233c47b415092c94f7c8b8d0bee [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:00.655895 7010 raft_consensus.cc:686] T 00000000000000000000000000000000 P cac0d233c47b415092c94f7c8b8d0bee [term 1 LEADER]: Becoming Leader. State: Replica: cac0d233c47b415092c94f7c8b8d0bee, State: Running, Role: LEADER
I20260810 07:19:00.656055 7000 sys_catalog.cc:564] T 00000000000000000000000000000000 P cac0d233c47b415092c94f7c8b8d0bee [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:00.656059 7010 consensus_queue.cc:237] T 00000000000000000000000000000000 P cac0d233c47b415092c94f7c8b8d0bee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cac0d233c47b415092c94f7c8b8d0bee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40977 } }
I20260810 07:19:00.656571 7013 sys_catalog.cc:455] T 00000000000000000000000000000000 P cac0d233c47b415092c94f7c8b8d0bee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cac0d233c47b415092c94f7c8b8d0bee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cac0d233c47b415092c94f7c8b8d0bee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40977 } } }
I20260810 07:19:00.656749 7013 sys_catalog.cc:458] T 00000000000000000000000000000000 P cac0d233c47b415092c94f7c8b8d0bee [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:00.656601 7014 sys_catalog.cc:455] T 00000000000000000000000000000000 P cac0d233c47b415092c94f7c8b8d0bee [sys.catalog]: SysCatalogTable state changed. Reason: New leader cac0d233c47b415092c94f7c8b8d0bee. Latest consensus state: current_term: 1 leader_uuid: "cac0d233c47b415092c94f7c8b8d0bee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cac0d233c47b415092c94f7c8b8d0bee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40977 } } }
I20260810 07:19:00.657013 7014 sys_catalog.cc:458] T 00000000000000000000000000000000 P cac0d233c47b415092c94f7c8b8d0bee [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:00.657120 7005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:00.655685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.141:55911 (local address 127.0.0.1:40977)
0810 07:19:00.655734 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:00.655737 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:00.655860 (+ 123us) server_negotiation.cc:407] Connection header received
0810 07:19:00.655975 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:00.655976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:00.655981 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:00.656007 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:00.656872 (+ 865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:00.656874 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:00.656874 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:00.656878 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:00.656895 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:00.656899 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:00.656929 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:00.657046 (+ 117us) server_negotiation.cc:299] Negotiation successful
0810 07:19:00.657062 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 07:19:00.657341 7022 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:00.657892 7022 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:00.662328 7022 catalog_manager.cc:1347] Generated new cluster ID: 6e649b8e372d48c9a9a72dc6b26bf6a3
I20260810 07:19:00.662392 7022 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:00.669548 7022 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:00.669744 7022 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:00.689718 7022 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cac0d233c47b415092c94f7c8b8d0bee: Generated new TSK 0
I20260810 07:19:00.689987 7022 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:19:00.697450 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 6926
2026-08-10T07:19:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1412 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:00 dist-test-slave-10pc krb5kdc[7035](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:00 dist-test-slave-10pc krb5kdc[7035](info): set up 2 sockets
Aug 10 07:19:00 dist-test-slave-10pc krb5kdc[7035](info): commencing operation
krb5kdc: starting...
W20260810 07:19:01.799782 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:19:01 dist-test-slave-10pc krb5kdc[7035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:19:01.927496 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43531
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43531
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:02.014016 7050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:02.014276 7050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:02.014321 7050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:02.015522 7050 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:02.015594 7050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:02.015625 7050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:02.015655 7050 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:02.015693 7050 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:02.017331 7050 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43531
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:02.017621 7050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:02.017884 7050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:19:02.023304 7050 server_base.cc:1034] running on GCE node
W20260810 07:19:02.023298 7055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:02.023443 7056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:02.023298 7058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:02.023815 7050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:02.024137 7050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:02.025313 7050 hybrid_clock.cc:648] HybridClock initialized: now 1786346342025293 us; error 39 us; skew 500 ppm
Aug 10 07:19:02 dist-test-slave-10pc krb5kdc[7035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346342, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:02.028057 7050 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:19:02.028560 7050 webserver.cc:460] Webserver started at http://127.0.0.1:36813/ using document root <none> and password file <none>
I20260810 07:19:02.028797 7050 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:02.028846 7050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:02.028955 7050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:02.030042 7050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "2c6abf39e2ea46b489bf26dc578cee3f"
format_stamp: "Formatted at 2026-08-10 07:19:02 on dist-test-slave-10pc"
server_key: "ae28fbe76ea5ce1330444c5ed23ec18b"
server_key_iv: "d6340ffbbaaa340cc6ecefc01a573549"
server_key_version: "encryptionkey@0"
I20260810 07:19:02.030391 7050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "2c6abf39e2ea46b489bf26dc578cee3f"
format_stamp: "Formatted at 2026-08-10 07:19:02 on dist-test-slave-10pc"
server_key: "ae28fbe76ea5ce1330444c5ed23ec18b"
server_key_iv: "d6340ffbbaaa340cc6ecefc01a573549"
server_key_version: "encryptionkey@0"
I20260810 07:19:02.031774 7050 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:19:02.032562 7066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:02.032749 7050 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:02.032851 7050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "2c6abf39e2ea46b489bf26dc578cee3f"
format_stamp: "Formatted at 2026-08-10 07:19:02 on dist-test-slave-10pc"
server_key: "ae28fbe76ea5ce1330444c5ed23ec18b"
server_key_iv: "d6340ffbbaaa340cc6ecefc01a573549"
server_key_version: "encryptionkey@0"
I20260810 07:19:02.032922 7050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:02.051685 7050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:02.054064 7050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:02.054204 7050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:02.058298 7050 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43531
I20260810 07:19:02.058384 7121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43531 every 8 connection(s)
I20260810 07:19:02.058756 7050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:02.059677 7122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:02.062229 7122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c6abf39e2ea46b489bf26dc578cee3f: Bootstrap starting.
I20260810 07:19:02.062862 7122 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c6abf39e2ea46b489bf26dc578cee3f: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:02.063189 7122 log.cc:824] T 00000000000000000000000000000000 P 2c6abf39e2ea46b489bf26dc578cee3f: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:02.063902 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 7050
I20260810 07:19:02.064019 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:19:02.064107 7122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c6abf39e2ea46b489bf26dc578cee3f: No bootstrap required, opened a new log
I20260810 07:19:02.064224 16229 external_mini_cluster.cc:1428] Setting key 8402d1cd448fe4391a6e6674f814eba1
I20260810 07:19:02.066766 7122 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c6abf39e2ea46b489bf26dc578cee3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c6abf39e2ea46b489bf26dc578cee3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43531 } }
I20260810 07:19:02.066941 7122 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c6abf39e2ea46b489bf26dc578cee3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:02.066990 7122 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c6abf39e2ea46b489bf26dc578cee3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c6abf39e2ea46b489bf26dc578cee3f, State: Initialized, Role: FOLLOWER
I20260810 07:19:02.067121 7122 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c6abf39e2ea46b489bf26dc578cee3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c6abf39e2ea46b489bf26dc578cee3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43531 } }
I20260810 07:19:02.067215 7122 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c6abf39e2ea46b489bf26dc578cee3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:02.067260 7122 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c6abf39e2ea46b489bf26dc578cee3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:02.067310 7122 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c6abf39e2ea46b489bf26dc578cee3f [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:02.067927 7122 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c6abf39e2ea46b489bf26dc578cee3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c6abf39e2ea46b489bf26dc578cee3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43531 } }
I20260810 07:19:02.068051 7122 leader_election.cc:302] T 00000000000000000000000000000000 P 2c6abf39e2ea46b489bf26dc578cee3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c6abf39e2ea46b489bf26dc578cee3f; no voters:
I20260810 07:19:02.068255 7122 leader_election.cc:288] T 00000000000000000000000000000000 P 2c6abf39e2ea46b489bf26dc578cee3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:02.068286 7127 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c6abf39e2ea46b489bf26dc578cee3f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:02.068487 7127 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c6abf39e2ea46b489bf26dc578cee3f [term 1 LEADER]: Becoming Leader. State: Replica: 2c6abf39e2ea46b489bf26dc578cee3f, State: Running, Role: LEADER
I20260810 07:19:02.068549 7122 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c6abf39e2ea46b489bf26dc578cee3f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:02.068620 7127 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c6abf39e2ea46b489bf26dc578cee3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c6abf39e2ea46b489bf26dc578cee3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43531 } }
I20260810 07:19:02.069049 7127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c6abf39e2ea46b489bf26dc578cee3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c6abf39e2ea46b489bf26dc578cee3f. Latest consensus state: current_term: 1 leader_uuid: "2c6abf39e2ea46b489bf26dc578cee3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c6abf39e2ea46b489bf26dc578cee3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43531 } } }
I20260810 07:19:02.069286 7127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c6abf39e2ea46b489bf26dc578cee3f [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:02.069209 7129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c6abf39e2ea46b489bf26dc578cee3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c6abf39e2ea46b489bf26dc578cee3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c6abf39e2ea46b489bf26dc578cee3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43531 } } }
I20260810 07:19:02.069766 7129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c6abf39e2ea46b489bf26dc578cee3f [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:02.069919 7137 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:02.070451 7137 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:02.072214 7137 catalog_manager.cc:1347] Generated new cluster ID: 39071014b508471ca4bd757f83cbae8f
I20260810 07:19:02.072273 7137 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 07:19:02 dist-test-slave-10pc krb5kdc[7035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 07:19:02.077740 7126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:02.064862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.141:55391 (local address 127.0.0.1:43531)
0810 07:19:02.065056 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:02.065066 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:02.065093 (+ 27us) server_negotiation.cc:407] Connection header received
0810 07:19:02.065708 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:02.065717 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:02.065775 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:02.065896 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:02.066744 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:02.068066 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:02.069240 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:02.069942 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:02.070033 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:02.073707 (+ 3674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:02.073712 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:02.073713 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:02.073727 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:02.075982 (+ 2255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:02.076518 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:02.076522 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:02.076522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:02.076622 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:02.076876 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:02.076877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:02.076877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:02.077138 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:02.077332 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:02.077448 (+ 116us) server_negotiation.cc:299] Negotiation successful
0810 07:19:02.077559 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":103,"threads_started":1}
I20260810 07:19:02.078742 16229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:19:02.086048 7137 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:02.086740 7137 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:02.087324 7126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:02.080887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.141:37417 (local address 127.0.0.1:43531)
0810 07:19:02.080929 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:02.080931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:02.081117 (+ 186us) server_negotiation.cc:407] Connection header received
0810 07:19:02.081193 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:02.081194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:02.081204 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:02.081260 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:02.081765 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:02.082271 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:02.083011 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:02.083239 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:02.083313 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:02.084548 (+ 1235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:02.084549 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:02.084549 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:02.084557 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:02.086155 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:02.086510 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:02.086511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:02.086512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:02.086568 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:02.086795 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:02.086796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:02.086797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:02.086926 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:02.087035 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:02.087164 (+ 129us) server_negotiation.cc:299] Negotiation successful
0810 07:19:02.087228 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 07:19:02.102476 7137 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c6abf39e2ea46b489bf26dc578cee3f: Generated new TSK 0
I20260810 07:19:02.102737 7137 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:19:02.110313 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 7050
W20260810 07:19:02.119052 7033 connection.cc:535] client connection to 127.0.0.1:43531 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43531): Connection reset by peer (error 104)
2026-08-10T07:19:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1413 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:02 dist-test-slave-10pc krb5kdc[7154](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:02 dist-test-slave-10pc krb5kdc[7154](info): set up 2 sockets
Aug 10 07:19:02 dist-test-slave-10pc krb5kdc[7154](info): commencing operation
krb5kdc: starting...
W20260810 07:19:03.233356 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:19:03 dist-test-slave-10pc krb5kdc[7154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:19:03.372258 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36809
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36809
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:03.461282 7169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:03.461547 7169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:03.461588 7169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:03.462785 7169 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:03.462853 7169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:03.462878 7169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:03.462910 7169 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:03.462934 7169 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:03.464535 7169 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36809
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:03.464820 7169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:03.465073 7169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:03.470296 7177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:03.470321 7174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:03.470305 7175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:03.470356 7169 server_base.cc:1034] running on GCE node
I20260810 07:19:03.470741 7169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:03.471020 7169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:03.472206 7169 hybrid_clock.cc:648] HybridClock initialized: now 1786346343472191 us; error 40 us; skew 500 ppm
Aug 10 07:19:03 dist-test-slave-10pc krb5kdc[7154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346343, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:03.475066 7169 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:19:03.475576 7169 webserver.cc:460] Webserver started at http://127.0.0.1:34611/ using document root <none> and password file <none>
I20260810 07:19:03.475824 7169 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:03.475891 7169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:03.476047 7169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:03.477293 7169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "6c06d0533d5d44a38c2e88f15c0a986c"
format_stamp: "Formatted at 2026-08-10 07:19:03 on dist-test-slave-10pc"
server_key: "56d5c7c4127ec29b187048cc7e11b8a1"
server_key_iv: "745223acdcfe9d5e6c6eb2eb1ee73f58"
server_key_version: "encryptionkey@0"
I20260810 07:19:03.477715 7169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "6c06d0533d5d44a38c2e88f15c0a986c"
format_stamp: "Formatted at 2026-08-10 07:19:03 on dist-test-slave-10pc"
server_key: "56d5c7c4127ec29b187048cc7e11b8a1"
server_key_iv: "745223acdcfe9d5e6c6eb2eb1ee73f58"
server_key_version: "encryptionkey@0"
I20260810 07:19:03.479555 7169 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:19:03.480407 7184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:03.480628 7169 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:03.480744 7169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "6c06d0533d5d44a38c2e88f15c0a986c"
format_stamp: "Formatted at 2026-08-10 07:19:03 on dist-test-slave-10pc"
server_key: "56d5c7c4127ec29b187048cc7e11b8a1"
server_key_iv: "745223acdcfe9d5e6c6eb2eb1ee73f58"
server_key_version: "encryptionkey@0"
I20260810 07:19:03.480826 7169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:03.494805 7169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:03.497349 7169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:03.497531 7169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:03.501989 7169 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36809
I20260810 07:19:03.502046 7241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36809 every 8 connection(s)
I20260810 07:19:03.502431 7169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:03.503360 7242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:03.505970 7242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c06d0533d5d44a38c2e88f15c0a986c: Bootstrap starting.
I20260810 07:19:03.506623 7242 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c06d0533d5d44a38c2e88f15c0a986c: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:03.506932 7242 log.cc:824] T 00000000000000000000000000000000 P 6c06d0533d5d44a38c2e88f15c0a986c: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:03.507683 7242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c06d0533d5d44a38c2e88f15c0a986c: No bootstrap required, opened a new log
I20260810 07:19:03.508873 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 7169
I20260810 07:19:03.509043 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:19:03.509248 16229 external_mini_cluster.cc:1428] Setting key 7cffedee3854e8b1325a62e6543b928b
I20260810 07:19:03.509750 7242 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c06d0533d5d44a38c2e88f15c0a986c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c06d0533d5d44a38c2e88f15c0a986c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36809 } }
I20260810 07:19:03.509969 7242 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c06d0533d5d44a38c2e88f15c0a986c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:03.510063 7242 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c06d0533d5d44a38c2e88f15c0a986c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c06d0533d5d44a38c2e88f15c0a986c, State: Initialized, Role: FOLLOWER
I20260810 07:19:03.510219 7242 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c06d0533d5d44a38c2e88f15c0a986c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c06d0533d5d44a38c2e88f15c0a986c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36809 } }
I20260810 07:19:03.510321 7242 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c06d0533d5d44a38c2e88f15c0a986c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:03.510367 7242 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c06d0533d5d44a38c2e88f15c0a986c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:03.510416 7242 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c06d0533d5d44a38c2e88f15c0a986c [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:03.511384 7242 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c06d0533d5d44a38c2e88f15c0a986c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c06d0533d5d44a38c2e88f15c0a986c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36809 } }
I20260810 07:19:03.511514 7242 leader_election.cc:302] T 00000000000000000000000000000000 P 6c06d0533d5d44a38c2e88f15c0a986c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c06d0533d5d44a38c2e88f15c0a986c; no voters:
I20260810 07:19:03.511716 7242 leader_election.cc:288] T 00000000000000000000000000000000 P 6c06d0533d5d44a38c2e88f15c0a986c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:03.511741 7247 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c06d0533d5d44a38c2e88f15c0a986c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:03.511857 7247 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c06d0533d5d44a38c2e88f15c0a986c [term 1 LEADER]: Becoming Leader. State: Replica: 6c06d0533d5d44a38c2e88f15c0a986c, State: Running, Role: LEADER
I20260810 07:19:03.512009 7242 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c06d0533d5d44a38c2e88f15c0a986c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:03.511999 7247 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c06d0533d5d44a38c2e88f15c0a986c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c06d0533d5d44a38c2e88f15c0a986c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36809 } }
I20260810 07:19:03.512555 7248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c06d0533d5d44a38c2e88f15c0a986c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c06d0533d5d44a38c2e88f15c0a986c. Latest consensus state: current_term: 1 leader_uuid: "6c06d0533d5d44a38c2e88f15c0a986c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c06d0533d5d44a38c2e88f15c0a986c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36809 } } }
I20260810 07:19:03.512655 7248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c06d0533d5d44a38c2e88f15c0a986c [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:03.512900 7249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c06d0533d5d44a38c2e88f15c0a986c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c06d0533d5d44a38c2e88f15c0a986c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c06d0533d5d44a38c2e88f15c0a986c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36809 } } }
I20260810 07:19:03.513262 7249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c06d0533d5d44a38c2e88f15c0a986c [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:03.513429 7254 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:03.513988 7254 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:03.515445 7246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:03.509846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.141:50333 (local address 127.0.0.1:36809)
0810 07:19:03.510097 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:03.510105 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:03.510148 (+ 43us) server_negotiation.cc:407] Connection header received
0810 07:19:03.510278 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:03.510285 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:03.510347 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:03.510490 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:03.510728 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:03.511762 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:03.512823 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:03.513443 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:03.513551 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:03.514662 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:03.514667 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:03.514667 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:03.514885 (+ 218us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:03.514927 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:03.514943 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:03.515048 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:03.515184 (+ 136us) server_negotiation.cc:299] Negotiation successful
0810 07:19:03.515218 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":113,"threads_started":1}
I20260810 07:19:03.515941 7254 catalog_manager.cc:1347] Generated new cluster ID: db5e5485837742c192c8b4c3dfe2bb50
I20260810 07:19:03.516002 7254 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:03.516599 16229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:19:03.521140 7246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:03.518167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.141:53257 (local address 127.0.0.1:36809)
0810 07:19:03.518204 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:03.518207 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:03.518234 (+ 27us) server_negotiation.cc:407] Connection header received
0810 07:19:03.518319 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:03.518320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:03.518331 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:03.518381 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:03.518521 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:03.519000 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:03.519757 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:03.520019 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:03.520094 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:03.520852 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:03.520853 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:03.520854 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:03.520859 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:03.520875 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:03.520880 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:03.520935 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:03.521009 (+ 74us) server_negotiation.cc:299] Negotiation successful
0810 07:19:03.521022 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260810 07:19:03.523804 7254 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:03.524401 7254 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:03.529611 7254 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c06d0533d5d44a38c2e88f15c0a986c: Generated new TSK 0
I20260810 07:19:03.529817 7254 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:19:03.534485 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 7169
W20260810 07:19:03.542831 7152 connection.cc:535] client connection to 127.0.0.1:36809 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36809): Connection reset by peer (error 104)
2026-08-10T07:19:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1423 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:03 dist-test-slave-10pc krb5kdc[7277](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:03 dist-test-slave-10pc krb5kdc[7277](info): set up 2 sockets
Aug 10 07:19:03 dist-test-slave-10pc krb5kdc[7277](info): commencing operation
krb5kdc: starting...
W20260810 07:19:04.636438 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:19:04 dist-test-slave-10pc krb5kdc[7277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346344, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:19:04.770982 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41635
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:04.857380 7290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:04.857641 7290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:04.857687 7290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:04.858886 7290 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:04.859040 7290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:04.859124 7290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:04.859155 7290 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:04.859184 7290 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:04.860929 7290 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41635
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346081759552-16229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:04.861223 7290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:04.861475 7290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:04.866771 7298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:04.866767 7300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:04.866793 7297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:04.867049 7290 server_base.cc:1034] running on GCE node
I20260810 07:19:04.867316 7290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:04.867633 7290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:04.868798 7290 hybrid_clock.cc:648] HybridClock initialized: now 1786346344868786 us; error 44 us; skew 500 ppm
Aug 10 07:19:04 dist-test-slave-10pc krb5kdc[7277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346344, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:04.871416 7290 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:19:04.871935 7290 webserver.cc:460] Webserver started at http://127.0.0.1:44299/ using document root <none> and password file <none>
I20260810 07:19:04.872169 7290 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:04.872238 7290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:04.872390 7290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:04.873554 7290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "f6bf069b350e4321ae4fcda5bd4a9242"
format_stamp: "Formatted at 2026-08-10 07:19:04 on dist-test-slave-10pc"
server_key: "3e89fa8159dce1d967780fbda64cc475"
server_key_iv: "8614968bead3ffae814f808f6612de61"
server_key_version: "encryptionkey@0"
I20260810 07:19:04.873957 7290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "f6bf069b350e4321ae4fcda5bd4a9242"
format_stamp: "Formatted at 2026-08-10 07:19:04 on dist-test-slave-10pc"
server_key: "3e89fa8159dce1d967780fbda64cc475"
server_key_iv: "8614968bead3ffae814f808f6612de61"
server_key_version: "encryptionkey@0"
I20260810 07:19:04.875361 7290 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:19:04.876169 7307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:04.876363 7290 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:04.876487 7290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "f6bf069b350e4321ae4fcda5bd4a9242"
format_stamp: "Formatted at 2026-08-10 07:19:04 on dist-test-slave-10pc"
server_key: "3e89fa8159dce1d967780fbda64cc475"
server_key_iv: "8614968bead3ffae814f808f6612de61"
server_key_version: "encryptionkey@0"
I20260810 07:19:04.876607 7290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:04.879999 7290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:04.882390 7290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:04.882553 7290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:04.886788 7290 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41635
I20260810 07:19:04.886822 7363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41635 every 8 connection(s)
I20260810 07:19:04.887178 7290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:04.888088 7364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:04.890581 7364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6bf069b350e4321ae4fcda5bd4a9242: Bootstrap starting.
I20260810 07:19:04.891233 7364 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6bf069b350e4321ae4fcda5bd4a9242: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:04.891557 7364 log.cc:824] T 00000000000000000000000000000000 P f6bf069b350e4321ae4fcda5bd4a9242: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:04.892284 7364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6bf069b350e4321ae4fcda5bd4a9242: No bootstrap required, opened a new log
I20260810 07:19:04.894003 7364 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6bf069b350e4321ae4fcda5bd4a9242 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6bf069b350e4321ae4fcda5bd4a9242" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41635 } }
I20260810 07:19:04.894153 7364 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6bf069b350e4321ae4fcda5bd4a9242 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:04.894181 7364 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6bf069b350e4321ae4fcda5bd4a9242 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6bf069b350e4321ae4fcda5bd4a9242, State: Initialized, Role: FOLLOWER
I20260810 07:19:04.894361 7364 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6bf069b350e4321ae4fcda5bd4a9242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6bf069b350e4321ae4fcda5bd4a9242" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41635 } }
I20260810 07:19:04.894429 7364 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6bf069b350e4321ae4fcda5bd4a9242 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:04.894481 7364 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6bf069b350e4321ae4fcda5bd4a9242 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:04.894547 7364 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6bf069b350e4321ae4fcda5bd4a9242 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:04.895140 7364 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6bf069b350e4321ae4fcda5bd4a9242 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6bf069b350e4321ae4fcda5bd4a9242" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41635 } }
I20260810 07:19:04.895258 7364 leader_election.cc:302] T 00000000000000000000000000000000 P f6bf069b350e4321ae4fcda5bd4a9242 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6bf069b350e4321ae4fcda5bd4a9242; no voters:
I20260810 07:19:04.895443 7364 leader_election.cc:288] T 00000000000000000000000000000000 P f6bf069b350e4321ae4fcda5bd4a9242 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:04.895473 7368 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6bf069b350e4321ae4fcda5bd4a9242 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:04.895653 7368 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6bf069b350e4321ae4fcda5bd4a9242 [term 1 LEADER]: Becoming Leader. State: Replica: f6bf069b350e4321ae4fcda5bd4a9242, State: Running, Role: LEADER
I20260810 07:19:04.895728 7364 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6bf069b350e4321ae4fcda5bd4a9242 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:04.895778 7368 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6bf069b350e4321ae4fcda5bd4a9242 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6bf069b350e4321ae4fcda5bd4a9242" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41635 } }
I20260810 07:19:04.896216 7370 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6bf069b350e4321ae4fcda5bd4a9242 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6bf069b350e4321ae4fcda5bd4a9242. Latest consensus state: current_term: 1 leader_uuid: "f6bf069b350e4321ae4fcda5bd4a9242" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6bf069b350e4321ae4fcda5bd4a9242" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41635 } } }
I20260810 07:19:04.896302 7370 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6bf069b350e4321ae4fcda5bd4a9242 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:04.896389 7369 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6bf069b350e4321ae4fcda5bd4a9242 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6bf069b350e4321ae4fcda5bd4a9242" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6bf069b350e4321ae4fcda5bd4a9242" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41635 } } }
I20260810 07:19:04.896512 7369 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6bf069b350e4321ae4fcda5bd4a9242 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:04.896808 7377 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:04.896936 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 7290
I20260810 07:19:04.897099 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:19:04.897321 16229 external_mini_cluster.cc:1428] Setting key 14a3d0ab73f6cbf34d5225978c66ee5f
I20260810 07:19:04.897457 7377 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:04.899452 7377 catalog_manager.cc:1347] Generated new cluster ID: 46eff27ff5444b1cb08a52ba3375176d
I20260810 07:19:04.899507 7377 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:04.899922 7390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:04.897909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.141:34273 (local address 127.0.0.1:41635)
0810 07:19:04.898116 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:04.898127 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:04.898231 (+ 104us) server_negotiation.cc:407] Connection header received
0810 07:19:04.898403 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:04.898411 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:04.898440 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:04.898600 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:04.899359 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:04.899363 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:04.899363 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:04.899368 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:04.899416 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:04.899432 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:04.899505 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:04.899602 (+ 97us) server_negotiation.cc:299] Negotiation successful
0810 07:19:04.899747 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":88,"threads_started":1}
I20260810 07:19:04.900885 16229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:19:04.903501 7390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:04.902341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.141:49537 (local address 127.0.0.1:41635)
0810 07:19:04.902382 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:04.902384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:04.902492 (+ 108us) server_negotiation.cc:407] Connection header received
0810 07:19:04.902581 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:04.902582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:04.902587 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:04.902615 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:04.903281 (+ 666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:04.903282 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:04.903282 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:04.903286 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:04.903299 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:04.903304 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:04.903332 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:04.903426 (+ 94us) server_negotiation.cc:299] Negotiation successful
0810 07:19:04.903438 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 07:19:04.905622 7377 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:04.905802 7377 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:04.910949 7377 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6bf069b350e4321ae4fcda5bd4a9242: Generated new TSK 0
I20260810 07:19:04.911157 7377 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:19:04.916396 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 7290
2026-08-10T07:19:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1382 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:04 dist-test-slave-10pc krb5kdc[7400](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:04 dist-test-slave-10pc krb5kdc[7400](info): set up 2 sockets
Aug 10 07:19:04 dist-test-slave-10pc krb5kdc[7400](info): commencing operation
krb5kdc: starting...
W20260810 07:19:06.036307 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:19:06 dist-test-slave-10pc krb5kdc[7400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
I20260810 07:19:06.172106 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:39905
--webserver_interface=127.15.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:35229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:39905
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:06.268191 7416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:06.268460 7416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:06.268532 7416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:06.269753 7416 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:06.269827 7416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:06.269851 7416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:06.269872 7416 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:06.269902 7416 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:06.271456 7416 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:35229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:39905
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:39905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:06.271754 7416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:06.272020 7416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:06.277078 7421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:06.277086 7423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:06.277107 7425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:06.277257 7416 server_base.cc:1034] running on GCE node
I20260810 07:19:06.277647 7416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:06.277969 7416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:06.279165 7416 hybrid_clock.cc:648] HybridClock initialized: now 1786346346279153 us; error 36 us; skew 500 ppm
Aug 10 07:19:06 dist-test-slave-10pc krb5kdc[7400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346346, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:06.286063 7416 init.cc:377] Logged in from keytab as kudu/127.15.217.126@KRBTEST.COM (short username kudu)
I20260810 07:19:06.286554 7416 webserver.cc:460] Webserver started at http://127.15.217.126:38265/ using document root <none> and password file <none>
I20260810 07:19:06.286783 7416 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:06.286827 7416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:06.286924 7416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:06.288029 7416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "3895b0a9bc124e2cb794fea005811610"
format_stamp: "Formatted at 2026-08-10 07:19:06 on dist-test-slave-10pc"
server_key: "cef93a66774c90619ba8ec63f8ae702c"
server_key_iv: "cf7bf5d94ee7da54d2e0a343c02bf221"
server_key_version: "encryptionkey@0"
I20260810 07:19:06.288347 7416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "3895b0a9bc124e2cb794fea005811610"
format_stamp: "Formatted at 2026-08-10 07:19:06 on dist-test-slave-10pc"
server_key: "cef93a66774c90619ba8ec63f8ae702c"
server_key_iv: "cf7bf5d94ee7da54d2e0a343c02bf221"
server_key_version: "encryptionkey@0"
I20260810 07:19:06.289783 7416 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:19:06.290567 7432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:06.290743 7416 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:06.290835 7416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "3895b0a9bc124e2cb794fea005811610"
format_stamp: "Formatted at 2026-08-10 07:19:06 on dist-test-slave-10pc"
server_key: "cef93a66774c90619ba8ec63f8ae702c"
server_key_iv: "cf7bf5d94ee7da54d2e0a343c02bf221"
server_key_version: "encryptionkey@0"
I20260810 07:19:06.290899 7416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:06.309140 7416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:06.311970 7416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:06.312136 7416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:06.316653 7416 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:39905
I20260810 07:19:06.316721 7487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:39905 every 8 connection(s)
I20260810 07:19:06.317116 7416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:06.318130 7488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:06.318943 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 7416
I20260810 07:19:06.319083 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:19:06.319279 16229 external_mini_cluster.cc:1428] Setting key e4d3104c5d66ba4bb182c649d2845a06
I20260810 07:19:06.321702 7488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3895b0a9bc124e2cb794fea005811610: Bootstrap starting.
I20260810 07:19:06.322374 7488 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3895b0a9bc124e2cb794fea005811610: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:06.322780 7488 log.cc:824] T 00000000000000000000000000000000 P 3895b0a9bc124e2cb794fea005811610: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:06.323585 7488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3895b0a9bc124e2cb794fea005811610: No bootstrap required, opened a new log
I20260810 07:19:06.326260 7488 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3895b0a9bc124e2cb794fea005811610 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3895b0a9bc124e2cb794fea005811610" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 39905 } }
I20260810 07:19:06.326460 7488 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3895b0a9bc124e2cb794fea005811610 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
Aug 10 07:19:06 dist-test-slave-10pc krb5kdc[7400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:19:06.326515 7488 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3895b0a9bc124e2cb794fea005811610 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3895b0a9bc124e2cb794fea005811610, State: Initialized, Role: FOLLOWER
I20260810 07:19:06.326686 7488 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3895b0a9bc124e2cb794fea005811610 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3895b0a9bc124e2cb794fea005811610" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 39905 } }
I20260810 07:19:06.326798 7488 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3895b0a9bc124e2cb794fea005811610 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:06.326843 7488 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3895b0a9bc124e2cb794fea005811610 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:06.326897 7488 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3895b0a9bc124e2cb794fea005811610 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:06.327669 7488 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3895b0a9bc124e2cb794fea005811610 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3895b0a9bc124e2cb794fea005811610" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 39905 } }
I20260810 07:19:06.327802 7488 leader_election.cc:302] T 00000000000000000000000000000000 P 3895b0a9bc124e2cb794fea005811610 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3895b0a9bc124e2cb794fea005811610; no voters:
I20260810 07:19:06.328037 7488 leader_election.cc:288] T 00000000000000000000000000000000 P 3895b0a9bc124e2cb794fea005811610 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:06.328100 7493 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3895b0a9bc124e2cb794fea005811610 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:06.328258 7493 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3895b0a9bc124e2cb794fea005811610 [term 1 LEADER]: Becoming Leader. State: Replica: 3895b0a9bc124e2cb794fea005811610, State: Running, Role: LEADER
I20260810 07:19:06.328378 7488 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3895b0a9bc124e2cb794fea005811610 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:06.328382 7493 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3895b0a9bc124e2cb794fea005811610 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3895b0a9bc124e2cb794fea005811610" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 39905 } }
I20260810 07:19:06.330071 7495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3895b0a9bc124e2cb794fea005811610 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3895b0a9bc124e2cb794fea005811610. Latest consensus state: current_term: 1 leader_uuid: "3895b0a9bc124e2cb794fea005811610" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3895b0a9bc124e2cb794fea005811610" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 39905 } } }
I20260810 07:19:06.330127 7494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3895b0a9bc124e2cb794fea005811610 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3895b0a9bc124e2cb794fea005811610" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3895b0a9bc124e2cb794fea005811610" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 39905 } } }
I20260810 07:19:06.330289 7494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3895b0a9bc124e2cb794fea005811610 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:06.330463 7495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3895b0a9bc124e2cb794fea005811610 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:06.330523 7509 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:06.331310 7509 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:06.331664 7491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:06.319867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32997 (local address 127.15.217.126:39905)
0810 07:19:06.320053 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:06.320063 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:06.320091 (+ 28us) server_negotiation.cc:407] Connection header received
0810 07:19:06.320650 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:06.320659 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:06.320715 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:06.320845 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:06.321731 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.322830 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.323784 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.324423 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.324581 (+ 158us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:06.327221 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:06.327226 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:06.327227 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:06.327247 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:06.329553 (+ 2306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:06.330044 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:06.330049 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:06.330050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:06.330204 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:06.330520 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:06.330522 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:06.330522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:06.330958 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:06.331146 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:06.331309 (+ 163us) server_negotiation.cc:299] Negotiation successful
0810 07:19:06.331420 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":95,"threads_started":1}
I20260810 07:19:06.332873 16229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:19:06.333148 7509 catalog_manager.cc:1347] Generated new cluster ID: fe67c91ceb8d4462b670d6dcfd56e0f8
I20260810 07:19:06.333206 7509 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:06.342407 7491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:06.334491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60793 (local address 127.15.217.126:39905)
0810 07:19:06.334537 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:06.334539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:06.334656 (+ 117us) server_negotiation.cc:407] Connection header received
0810 07:19:06.334754 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:06.334755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:06.334764 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:06.334790 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:06.335473 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.336221 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.337265 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.337601 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.337639 (+ 38us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:06.339220 (+ 1581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:06.339222 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:06.339222 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:06.339230 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:06.340990 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:06.341488 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:06.341489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:06.341489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:06.341553 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:06.341805 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:06.341805 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:06.341806 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:06.341951 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:06.342049 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:06.342228 (+ 179us) server_negotiation.cc:299] Negotiation successful
0810 07:19:06.342303 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 07:19:06.349687 7509 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:06.350513 7509 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:06.356741 7509 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3895b0a9bc124e2cb794fea005811610: Generated new TSK 0
I20260810 07:19:06.357000 7509 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:19:06.359790 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 7416
W20260810 07:19:06.367225 7398 connection.cc:535] client connection to 127.15.217.126:39905 recv error: Network error: failed to read from TLS socket (remote: 127.15.217.126:39905): Connection reset by peer (error 104)
2026-08-10T07:19:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1441 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:06 dist-test-slave-10pc krb5kdc[7520](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:06 dist-test-slave-10pc krb5kdc[7520](info): set up 2 sockets
Aug 10 07:19:06 dist-test-slave-10pc krb5kdc[7520](info): commencing operation
krb5kdc: starting...
W20260810 07:19:07.461009 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:19:07 dist-test-slave-10pc krb5kdc[7520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
I20260810 07:19:07.593631 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:39443
--webserver_interface=127.15.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:45785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:39443
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:07.681922 7535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:07.682201 7535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:07.682243 7535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:07.683441 7535 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:07.683518 7535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:07.683542 7535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:07.683564 7535 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:07.683595 7535 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:07.685180 7535 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:45785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:39443
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:39443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:07.685496 7535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:07.685753 7535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:07.691039 7543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:07.691191 7535 server_base.cc:1034] running on GCE node
W20260810 07:19:07.691195 7542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:07.691039 7547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:07.691546 7535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:07.691908 7535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:07.693104 7535 hybrid_clock.cc:648] HybridClock initialized: now 1786346347693077 us; error 37 us; skew 500 ppm
Aug 10 07:19:07 dist-test-slave-10pc krb5kdc[7520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346347, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:07.695926 7535 init.cc:377] Logged in from keytab as kudu/127.15.217.126@KRBTEST.COM (short username kudu)
I20260810 07:19:07.696600 7535 webserver.cc:460] Webserver started at http://127.15.217.126:44681/ using document root <none> and password file <none>
I20260810 07:19:07.696877 7535 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:07.696955 7535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:07.697129 7535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:07.698470 7535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "7a7f7dc10f3147888c00fd0478b13c2c"
format_stamp: "Formatted at 2026-08-10 07:19:07 on dist-test-slave-10pc"
server_key: "a2f29c47234fa8ac568e1327bddf2367"
server_key_iv: "6c993afc05d3c401d747598497f78dbb"
server_key_version: "encryptionkey@0"
I20260810 07:19:07.698921 7535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "7a7f7dc10f3147888c00fd0478b13c2c"
format_stamp: "Formatted at 2026-08-10 07:19:07 on dist-test-slave-10pc"
server_key: "a2f29c47234fa8ac568e1327bddf2367"
server_key_iv: "6c993afc05d3c401d747598497f78dbb"
server_key_version: "encryptionkey@0"
I20260810 07:19:07.700600 7535 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260810 07:19:07.701471 7555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:07.701666 7535 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:07.701792 7535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "7a7f7dc10f3147888c00fd0478b13c2c"
format_stamp: "Formatted at 2026-08-10 07:19:07 on dist-test-slave-10pc"
server_key: "a2f29c47234fa8ac568e1327bddf2367"
server_key_iv: "6c993afc05d3c401d747598497f78dbb"
server_key_version: "encryptionkey@0"
I20260810 07:19:07.701880 7535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:07.727363 7535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:07.729513 7535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:07.729701 7535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:07.734228 7535 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:39443
I20260810 07:19:07.734267 7609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:39443 every 8 connection(s)
I20260810 07:19:07.734692 7535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:07.735652 7610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:07.738217 7610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a7f7dc10f3147888c00fd0478b13c2c: Bootstrap starting.
I20260810 07:19:07.738865 7610 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a7f7dc10f3147888c00fd0478b13c2c: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:07.739195 7610 log.cc:824] T 00000000000000000000000000000000 P 7a7f7dc10f3147888c00fd0478b13c2c: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:07.740085 7610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a7f7dc10f3147888c00fd0478b13c2c: No bootstrap required, opened a new log
I20260810 07:19:07.740185 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 7535
I20260810 07:19:07.740334 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:19:07.740602 16229 external_mini_cluster.cc:1428] Setting key 88d8b66d096582867ca4390d97f5094d
I20260810 07:19:07.742298 7610 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a7f7dc10f3147888c00fd0478b13c2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a7f7dc10f3147888c00fd0478b13c2c" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 39443 } }
I20260810 07:19:07.742479 7610 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a7f7dc10f3147888c00fd0478b13c2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:07.742574 7610 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a7f7dc10f3147888c00fd0478b13c2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a7f7dc10f3147888c00fd0478b13c2c, State: Initialized, Role: FOLLOWER
I20260810 07:19:07.742749 7610 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a7f7dc10f3147888c00fd0478b13c2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a7f7dc10f3147888c00fd0478b13c2c" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 39443 } }
I20260810 07:19:07.742851 7610 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a7f7dc10f3147888c00fd0478b13c2c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:07.742877 7610 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a7f7dc10f3147888c00fd0478b13c2c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:07.742950 7610 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a7f7dc10f3147888c00fd0478b13c2c [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:07.743602 7610 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a7f7dc10f3147888c00fd0478b13c2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a7f7dc10f3147888c00fd0478b13c2c" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 39443 } }
I20260810 07:19:07.743716 7610 leader_election.cc:302] T 00000000000000000000000000000000 P 7a7f7dc10f3147888c00fd0478b13c2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a7f7dc10f3147888c00fd0478b13c2c; no voters:
I20260810 07:19:07.743957 7610 leader_election.cc:288] T 00000000000000000000000000000000 P 7a7f7dc10f3147888c00fd0478b13c2c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:07.743981 7615 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a7f7dc10f3147888c00fd0478b13c2c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:07.744112 7615 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a7f7dc10f3147888c00fd0478b13c2c [term 1 LEADER]: Becoming Leader. State: Replica: 7a7f7dc10f3147888c00fd0478b13c2c, State: Running, Role: LEADER
I20260810 07:19:07.744269 7610 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a7f7dc10f3147888c00fd0478b13c2c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:07.744267 7615 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a7f7dc10f3147888c00fd0478b13c2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a7f7dc10f3147888c00fd0478b13c2c" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 39443 } }
I20260810 07:19:07.744786 7617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a7f7dc10f3147888c00fd0478b13c2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a7f7dc10f3147888c00fd0478b13c2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a7f7dc10f3147888c00fd0478b13c2c" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 39443 } } }
I20260810 07:19:07.745074 7617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a7f7dc10f3147888c00fd0478b13c2c [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:07.744987 7616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a7f7dc10f3147888c00fd0478b13c2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a7f7dc10f3147888c00fd0478b13c2c. Latest consensus state: current_term: 1 leader_uuid: "7a7f7dc10f3147888c00fd0478b13c2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a7f7dc10f3147888c00fd0478b13c2c" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 39443 } } }
I20260810 07:19:07.745565 7616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a7f7dc10f3147888c00fd0478b13c2c [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:07.745571 7624 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:07.746208 7624 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:07.748068 7624 catalog_manager.cc:1347] Generated new cluster ID: 6964aaf679ae4acbb0459953c8f7f6ad
I20260810 07:19:07.748126 7624 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:07.748494 7614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:07.741243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59995 (local address 127.15.217.126:39443)
0810 07:19:07.741437 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:07.741446 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:07.741487 (+ 41us) server_negotiation.cc:407] Connection header received
0810 07:19:07.742279 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:07.742289 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:07.742352 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:07.742538 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:07.742802 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.744450 (+ 1648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.745659 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.746299 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.746409 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:07.747540 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:07.747543 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:07.747544 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:07.747787 (+ 243us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:07.747835 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:07.747851 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:07.747949 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:07.748206 (+ 257us) server_negotiation.cc:299] Negotiation successful
0810 07:19:07.748244 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":100,"threads_started":1}
I20260810 07:19:07.749639 16229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:19:07.754360 7624 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:07.754875 7624 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:07.755326 7614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:07.751252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37299 (local address 127.15.217.126:39443)
0810 07:19:07.751320 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:07.751323 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:07.751568 (+ 245us) server_negotiation.cc:407] Connection header received
0810 07:19:07.751631 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:07.751632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:07.751643 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:07.751696 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:07.751843 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.752637 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.753767 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.754049 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.754129 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:07.755012 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:07.755013 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:07.755014 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:07.755020 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:07.755035 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:07.755040 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:07.755073 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:07.755232 (+ 159us) server_negotiation.cc:299] Negotiation successful
0810 07:19:07.755247 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 07:19:07.757225 7624 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a7f7dc10f3147888c00fd0478b13c2c: Generated new TSK 0
I20260810 07:19:07.757427 7624 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:19:07.761395 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 7535
W20260810 07:19:07.770443 7517 connection.cc:535] client connection to 127.15.217.126:39443 recv error: Network error: failed to read from TLS socket (remote: 127.15.217.126:39443): Connection reset by peer (error 104)
2026-08-10T07:19:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1403 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:07 dist-test-slave-10pc krb5kdc[7643](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:07 dist-test-slave-10pc krb5kdc[7643](info): set up 2 sockets
Aug 10 07:19:07 dist-test-slave-10pc krb5kdc[7643](info): commencing operation
krb5kdc: starting...
W20260810 07:19:08.851742 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:19:08 dist-test-slave-10pc krb5kdc[7643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
I20260810 07:19:08.982997 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:36121
--webserver_interface=127.15.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:38251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:36121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:09.072676 7657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:09.072897 7657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:09.072921 7657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:09.074124 7657 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:09.074169 7657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:09.074183 7657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:09.074196 7657 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:09.074208 7657 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:09.075747 7657 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:38251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:36121
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:36121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:09.076097 7657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:09.076375 7657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:19:09.081595 7657 server_base.cc:1034] running on GCE node
W20260810 07:19:09.081550 7662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:09.081550 7665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:09.081538 7663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:09.082176 7657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:09.082489 7657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:09.083668 7657 hybrid_clock.cc:648] HybridClock initialized: now 1786346349083648 us; error 38 us; skew 500 ppm
Aug 10 07:19:09 dist-test-slave-10pc krb5kdc[7643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346349, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:09.086299 7657 init.cc:377] Logged in from keytab as kudu/127.15.217.126@KRBTEST.COM (short username kudu)
I20260810 07:19:09.086738 7657 webserver.cc:460] Webserver started at http://127.15.217.126:45789/ using document root <none> and password file <none>
I20260810 07:19:09.086962 7657 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:09.087030 7657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:09.087183 7657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:09.088310 7657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "876cb52f716947a89385101bb0ab3f98"
format_stamp: "Formatted at 2026-08-10 07:19:09 on dist-test-slave-10pc"
server_key: "7d0b5769eda30aaeffc78b367ea654f9"
server_key_iv: "5c9885708086e37b0f425bf82a7ba92a"
server_key_version: "encryptionkey@0"
I20260810 07:19:09.088718 7657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "876cb52f716947a89385101bb0ab3f98"
format_stamp: "Formatted at 2026-08-10 07:19:09 on dist-test-slave-10pc"
server_key: "7d0b5769eda30aaeffc78b367ea654f9"
server_key_iv: "5c9885708086e37b0f425bf82a7ba92a"
server_key_version: "encryptionkey@0"
I20260810 07:19:09.090241 7657 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:09.090976 7677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:09.091187 7657 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:09.091289 7657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "876cb52f716947a89385101bb0ab3f98"
format_stamp: "Formatted at 2026-08-10 07:19:09 on dist-test-slave-10pc"
server_key: "7d0b5769eda30aaeffc78b367ea654f9"
server_key_iv: "5c9885708086e37b0f425bf82a7ba92a"
server_key_version: "encryptionkey@0"
I20260810 07:19:09.091372 7657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:09.094799 7657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:09.096773 7657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:09.096922 7657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:09.101169 7657 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:36121
I20260810 07:19:09.101207 7733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:36121 every 8 connection(s)
I20260810 07:19:09.101570 7657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:09.102895 7734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:09.106144 7734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 876cb52f716947a89385101bb0ab3f98: Bootstrap starting.
I20260810 07:19:09.106779 7734 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 876cb52f716947a89385101bb0ab3f98: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:09.107093 7734 log.cc:824] T 00000000000000000000000000000000 P 876cb52f716947a89385101bb0ab3f98: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:09.107825 7734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 876cb52f716947a89385101bb0ab3f98: No bootstrap required, opened a new log
I20260810 07:19:09.109468 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 7657
I20260810 07:19:09.109571 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:19:09.109798 16229 external_mini_cluster.cc:1428] Setting key 57217d43c7892084d5eda11c548c7ed3
I20260810 07:19:09.109748 7734 raft_consensus.cc:348] T 00000000000000000000000000000000 P 876cb52f716947a89385101bb0ab3f98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "876cb52f716947a89385101bb0ab3f98" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 36121 } }
I20260810 07:19:09.109951 7734 raft_consensus.cc:374] T 00000000000000000000000000000000 P 876cb52f716947a89385101bb0ab3f98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:09.110009 7734 raft_consensus.cc:729] T 00000000000000000000000000000000 P 876cb52f716947a89385101bb0ab3f98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 876cb52f716947a89385101bb0ab3f98, State: Initialized, Role: FOLLOWER
I20260810 07:19:09.110183 7734 consensus_queue.cc:260] T 00000000000000000000000000000000 P 876cb52f716947a89385101bb0ab3f98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "876cb52f716947a89385101bb0ab3f98" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 36121 } }
I20260810 07:19:09.110271 7734 raft_consensus.cc:388] T 00000000000000000000000000000000 P 876cb52f716947a89385101bb0ab3f98 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:09.110306 7734 raft_consensus.cc:482] T 00000000000000000000000000000000 P 876cb52f716947a89385101bb0ab3f98 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:09.110349 7734 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 876cb52f716947a89385101bb0ab3f98 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:09.110989 7734 raft_consensus.cc:504] T 00000000000000000000000000000000 P 876cb52f716947a89385101bb0ab3f98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "876cb52f716947a89385101bb0ab3f98" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 36121 } }
I20260810 07:19:09.111119 7734 leader_election.cc:302] T 00000000000000000000000000000000 P 876cb52f716947a89385101bb0ab3f98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 876cb52f716947a89385101bb0ab3f98; no voters:
I20260810 07:19:09.111305 7734 leader_election.cc:288] T 00000000000000000000000000000000 P 876cb52f716947a89385101bb0ab3f98 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:09.111330 7739 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 876cb52f716947a89385101bb0ab3f98 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:09.111637 7734 sys_catalog.cc:564] T 00000000000000000000000000000000 P 876cb52f716947a89385101bb0ab3f98 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:09.111951 7739 raft_consensus.cc:686] T 00000000000000000000000000000000 P 876cb52f716947a89385101bb0ab3f98 [term 1 LEADER]: Becoming Leader. State: Replica: 876cb52f716947a89385101bb0ab3f98, State: Running, Role: LEADER
I20260810 07:19:09.112058 7739 consensus_queue.cc:237] T 00000000000000000000000000000000 P 876cb52f716947a89385101bb0ab3f98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "876cb52f716947a89385101bb0ab3f98" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 36121 } }
I20260810 07:19:09.112490 7738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:09.110347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54037 (local address 127.15.217.126:36121)
0810 07:19:09.110555 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:09.110565 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:09.110622 (+ 57us) server_negotiation.cc:407] Connection header received
0810 07:19:09.110997 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:09.111003 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:09.111028 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:09.111185 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:09.111975 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:09.111979 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:09.111979 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:09.111984 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:09.112033 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:09.112048 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:09.112110 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:09.112189 (+ 79us) server_negotiation.cc:299] Negotiation successful
0810 07:19:09.112237 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":98,"threads_started":1}
I20260810 07:19:09.112747 7740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 876cb52f716947a89385101bb0ab3f98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "876cb52f716947a89385101bb0ab3f98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "876cb52f716947a89385101bb0ab3f98" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 36121 } } }
I20260810 07:19:09.112835 7740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 876cb52f716947a89385101bb0ab3f98 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:09.112972 7740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 876cb52f716947a89385101bb0ab3f98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 876cb52f716947a89385101bb0ab3f98. Latest consensus state: current_term: 1 leader_uuid: "876cb52f716947a89385101bb0ab3f98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "876cb52f716947a89385101bb0ab3f98" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 36121 } } }
I20260810 07:19:09.113039 7740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 876cb52f716947a89385101bb0ab3f98 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:09.113598 16229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260810 07:19:09.114044 7757 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 876cb52f716947a89385101bb0ab3f98: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:19:09.114090 7757 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:19:09.114171 7748 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:09.114389 7748 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:09.116020 7748 catalog_manager.cc:1347] Generated new cluster ID: 3832c44aedf24c768c8b3f2adb1ded50
I20260810 07:19:09.116080 7748 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:09.116894 7738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:09.115236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39421 (local address 127.15.217.126:36121)
0810 07:19:09.115271 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:09.115273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:09.115321 (+ 48us) server_negotiation.cc:407] Connection header received
0810 07:19:09.115394 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:09.115395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:09.115400 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:09.115458 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:09.116669 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:09.116670 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:09.116671 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:09.116675 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:09.116691 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:09.116696 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:09.116762 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:09.116819 (+ 57us) server_negotiation.cc:299] Negotiation successful
0810 07:19:09.116833 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 07:19:09.136106 7748 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:09.136334 7748 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:09.146421 7748 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 876cb52f716947a89385101bb0ab3f98: Generated new TSK 0
I20260810 07:19:09.146713 7748 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:19:09.156613 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 7657
2026-08-10T07:19:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1393 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:09 dist-test-slave-10pc krb5kdc[7767](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:09 dist-test-slave-10pc krb5kdc[7767](info): set up 2 sockets
Aug 10 07:19:09 dist-test-slave-10pc krb5kdc[7767](info): commencing operation
krb5kdc: starting...
W20260810 07:19:10.253564 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:19:10 dist-test-slave-10pc krb5kdc[7767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
I20260810 07:19:10.387878 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:46143
--webserver_interface=127.15.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:39881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:46143
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:10.477804 7780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:10.478034 7780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:10.478058 7780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:10.479168 7780 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:10.479205 7780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:10.479220 7780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:10.479233 7780 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:10.479245 7780 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:10.480772 7780 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:39881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:46143
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:46143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:10.480990 7780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:10.481174 7780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:10.486171 7786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:10.486202 7788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:10.486200 7785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:10.486516 7780 server_base.cc:1034] running on GCE node
I20260810 07:19:10.486817 7780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:10.487130 7780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:10.488312 7780 hybrid_clock.cc:648] HybridClock initialized: now 1786346350488285 us; error 44 us; skew 500 ppm
Aug 10 07:19:10 dist-test-slave-10pc krb5kdc[7767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346350, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:10.491056 7780 init.cc:377] Logged in from keytab as kudu/127.15.217.126@KRBTEST.COM (short username kudu)
I20260810 07:19:10.491534 7780 webserver.cc:460] Webserver started at http://127.15.217.126:39161/ using document root <none> and password file <none>
I20260810 07:19:10.491779 7780 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:10.491849 7780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:10.492008 7780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:10.493237 7780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "0cc383065c9a41938d7d1a6dd0327d14"
format_stamp: "Formatted at 2026-08-10 07:19:10 on dist-test-slave-10pc"
server_key: "cd1decc6c8d7bf785cd58049c683e3ae"
server_key_iv: "4a3f7fdf155ade286fe0811dfdfa8953"
server_key_version: "encryptionkey@0"
I20260810 07:19:10.493630 7780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "0cc383065c9a41938d7d1a6dd0327d14"
format_stamp: "Formatted at 2026-08-10 07:19:10 on dist-test-slave-10pc"
server_key: "cd1decc6c8d7bf785cd58049c683e3ae"
server_key_iv: "4a3f7fdf155ade286fe0811dfdfa8953"
server_key_version: "encryptionkey@0"
I20260810 07:19:10.495189 7780 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 07:19:10.495995 7796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:10.496212 7780 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:10.496326 7780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "0cc383065c9a41938d7d1a6dd0327d14"
format_stamp: "Formatted at 2026-08-10 07:19:10 on dist-test-slave-10pc"
server_key: "cd1decc6c8d7bf785cd58049c683e3ae"
server_key_iv: "4a3f7fdf155ade286fe0811dfdfa8953"
server_key_version: "encryptionkey@0"
I20260810 07:19:10.496405 7780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:10.506044 7780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:10.507987 7780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:10.508168 7780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:10.512219 7780 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:46143
I20260810 07:19:10.512270 7858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:46143 every 8 connection(s)
I20260810 07:19:10.512704 7780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:10.513660 7859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:10.514429 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 7780
I20260810 07:19:10.514580 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:19:10.514794 16229 external_mini_cluster.cc:1428] Setting key e737c6ece2fd955276ffaa63eca9c984
I20260810 07:19:10.517031 7859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cc383065c9a41938d7d1a6dd0327d14: Bootstrap starting.
I20260810 07:19:10.517809 7859 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0cc383065c9a41938d7d1a6dd0327d14: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:10.518180 7859 log.cc:824] T 00000000000000000000000000000000 P 0cc383065c9a41938d7d1a6dd0327d14: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:10.518937 7859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cc383065c9a41938d7d1a6dd0327d14: No bootstrap required, opened a new log
I20260810 07:19:10.521414 7859 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0cc383065c9a41938d7d1a6dd0327d14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cc383065c9a41938d7d1a6dd0327d14" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 46143 } }
I20260810 07:19:10.521564 7859 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0cc383065c9a41938d7d1a6dd0327d14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:10.521585 7859 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0cc383065c9a41938d7d1a6dd0327d14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cc383065c9a41938d7d1a6dd0327d14, State: Initialized, Role: FOLLOWER
I20260810 07:19:10.521759 7859 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0cc383065c9a41938d7d1a6dd0327d14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cc383065c9a41938d7d1a6dd0327d14" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 46143 } }
I20260810 07:19:10.521860 7859 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0cc383065c9a41938d7d1a6dd0327d14 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:10.521910 7859 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0cc383065c9a41938d7d1a6dd0327d14 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:10.521968 7859 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0cc383065c9a41938d7d1a6dd0327d14 [term 0 FOLLOWER]: Advancing to term 1
Aug 10 07:19:10 dist-test-slave-10pc krb5kdc[7767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:19:10.522637 7859 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0cc383065c9a41938d7d1a6dd0327d14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cc383065c9a41938d7d1a6dd0327d14" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 46143 } }
I20260810 07:19:10.522778 7859 leader_election.cc:302] T 00000000000000000000000000000000 P 0cc383065c9a41938d7d1a6dd0327d14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0cc383065c9a41938d7d1a6dd0327d14; no voters:
I20260810 07:19:10.522964 7859 leader_election.cc:288] T 00000000000000000000000000000000 P 0cc383065c9a41938d7d1a6dd0327d14 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:10.523016 7864 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0cc383065c9a41938d7d1a6dd0327d14 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:10.523200 7864 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0cc383065c9a41938d7d1a6dd0327d14 [term 1 LEADER]: Becoming Leader. State: Replica: 0cc383065c9a41938d7d1a6dd0327d14, State: Running, Role: LEADER
I20260810 07:19:10.523279 7859 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0cc383065c9a41938d7d1a6dd0327d14 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:10.523340 7864 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0cc383065c9a41938d7d1a6dd0327d14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cc383065c9a41938d7d1a6dd0327d14" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 46143 } }
I20260810 07:19:10.523756 7864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cc383065c9a41938d7d1a6dd0327d14 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0cc383065c9a41938d7d1a6dd0327d14. Latest consensus state: current_term: 1 leader_uuid: "0cc383065c9a41938d7d1a6dd0327d14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cc383065c9a41938d7d1a6dd0327d14" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 46143 } } }
I20260810 07:19:10.523984 7864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cc383065c9a41938d7d1a6dd0327d14 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:10.523908 7866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cc383065c9a41938d7d1a6dd0327d14 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0cc383065c9a41938d7d1a6dd0327d14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cc383065c9a41938d7d1a6dd0327d14" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 46143 } } }
I20260810 07:19:10.524385 7866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cc383065c9a41938d7d1a6dd0327d14 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:10.524626 7875 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:10.525285 7875 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:10.526979 7875 catalog_manager.cc:1347] Generated new cluster ID: 16e2c3fdfdbd48049656de8b9eacf98d
I20260810 07:19:10.527035 7875 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:10.527273 7862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:10.515416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45481 (local address 127.15.217.126:46143)
0810 07:19:10.515641 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:10.515651 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:10.515684 (+ 33us) server_negotiation.cc:407] Connection header received
0810 07:19:10.516198 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:10.516208 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:10.516276 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:10.516433 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:10.517202 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:10.518667 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:10.519459 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:10.520088 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:10.520214 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:10.522761 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:10.522766 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:10.522766 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:10.522779 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:10.525175 (+ 2396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:10.525605 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:10.525615 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:10.525616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:10.525728 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:10.525969 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:10.525970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:10.525970 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:10.526291 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:10.526476 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:10.526904 (+ 428us) server_negotiation.cc:299] Negotiation successful
0810 07:19:10.527064 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":104,"threads_started":1}
I20260810 07:19:10.528337 16229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:19:10.537674 7862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:10.529967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52713 (local address 127.15.217.126:46143)
0810 07:19:10.530003 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:10.530005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:10.530012 (+ 7us) server_negotiation.cc:407] Connection header received
0810 07:19:10.530038 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:10.530039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:10.530049 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:10.530118 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:10.530717 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:10.531832 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:10.532840 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:10.533181 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:10.533262 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:10.534680 (+ 1418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:10.534681 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:10.534682 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:10.534689 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:10.536334 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:10.536756 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:10.536757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:10.536757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:10.536822 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:10.537021 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:10.537022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:10.537022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:10.537163 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:10.537267 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:10.537374 (+ 107us) server_negotiation.cc:299] Negotiation successful
0810 07:19:10.537457 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 07:19:10.545763 7875 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:10.546337 7875 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:10.550355 7875 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0cc383065c9a41938d7d1a6dd0327d14: Generated new TSK 0
I20260810 07:19:10.550561 7875 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:19:10.552856 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 7780
2026-08-10T07:19:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1396 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:10 dist-test-slave-10pc krb5kdc[7893](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:10 dist-test-slave-10pc krb5kdc[7893](info): set up 2 sockets
Aug 10 07:19:10 dist-test-slave-10pc krb5kdc[7893](info): commencing operation
krb5kdc: starting...
W20260810 07:19:11.642932 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:19:11 dist-test-slave-10pc krb5kdc[7893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
I20260810 07:19:11.779776 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:44859
--webserver_interface=127.15.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:44199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:44859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:11.866698 7906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:11.866961 7906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:11.867007 7906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:11.868765 7906 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:11.868842 7906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:11.868867 7906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:11.868896 7906 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:11.868927 7906 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:11.870477 7906 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:44199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:44859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:44859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:11.870787 7906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:11.871039 7906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:11.876214 7915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:11.876384 7913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:11.876212 7911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:11.876566 7906 server_base.cc:1034] running on GCE node
I20260810 07:19:11.876834 7906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:11.877110 7906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:11.878342 7906 hybrid_clock.cc:648] HybridClock initialized: now 1786346351878324 us; error 28 us; skew 500 ppm
Aug 10 07:19:11 dist-test-slave-10pc krb5kdc[7893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346351, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:11.886147 7906 init.cc:377] Logged in from keytab as kudu/127.15.217.126@KRBTEST.COM (short username kudu)
I20260810 07:19:11.886752 7906 webserver.cc:460] Webserver started at http://127.15.217.126:46323/ using document root <none> and password file <none>
I20260810 07:19:11.887025 7906 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:11.887099 7906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:11.887261 7906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:11.888540 7906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "69a4acdbe7dd44328f562313385732d7"
format_stamp: "Formatted at 2026-08-10 07:19:11 on dist-test-slave-10pc"
server_key: "d476cef29cbe08bcb8c977d2cc50d0be"
server_key_iv: "ca1322cce5b3e07ae16688d4d88d0c02"
server_key_version: "encryptionkey@0"
I20260810 07:19:11.888968 7906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "69a4acdbe7dd44328f562313385732d7"
format_stamp: "Formatted at 2026-08-10 07:19:11 on dist-test-slave-10pc"
server_key: "d476cef29cbe08bcb8c977d2cc50d0be"
server_key_iv: "ca1322cce5b3e07ae16688d4d88d0c02"
server_key_version: "encryptionkey@0"
I20260810 07:19:11.890516 7906 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 07:19:11.891402 7923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:11.891629 7906 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:19:11.891762 7906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "69a4acdbe7dd44328f562313385732d7"
format_stamp: "Formatted at 2026-08-10 07:19:11 on dist-test-slave-10pc"
server_key: "d476cef29cbe08bcb8c977d2cc50d0be"
server_key_iv: "ca1322cce5b3e07ae16688d4d88d0c02"
server_key_version: "encryptionkey@0"
I20260810 07:19:11.891848 7906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:11.924552 7906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:11.927270 7906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:11.927467 7906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:11.932181 7906 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:44859
I20260810 07:19:11.932222 7981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:44859 every 8 connection(s)
I20260810 07:19:11.932685 7906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:11.933610 7982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:11.936102 7982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69a4acdbe7dd44328f562313385732d7: Bootstrap starting.
I20260810 07:19:11.936439 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 7906
I20260810 07:19:11.936599 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:19:11.936821 16229 external_mini_cluster.cc:1428] Setting key fe5ce4d8b694229692e35df8e67afa94
I20260810 07:19:11.936894 7982 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69a4acdbe7dd44328f562313385732d7: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:11.937247 7982 log.cc:824] T 00000000000000000000000000000000 P 69a4acdbe7dd44328f562313385732d7: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:11.938099 7982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69a4acdbe7dd44328f562313385732d7: No bootstrap required, opened a new log
I20260810 07:19:11.940058 7982 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69a4acdbe7dd44328f562313385732d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a4acdbe7dd44328f562313385732d7" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 44859 } }
I20260810 07:19:11.940231 7982 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69a4acdbe7dd44328f562313385732d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:11.940291 7982 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69a4acdbe7dd44328f562313385732d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69a4acdbe7dd44328f562313385732d7, State: Initialized, Role: FOLLOWER
I20260810 07:19:11.940478 7982 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69a4acdbe7dd44328f562313385732d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a4acdbe7dd44328f562313385732d7" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 44859 } }
I20260810 07:19:11.940594 7982 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69a4acdbe7dd44328f562313385732d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:11.940642 7982 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69a4acdbe7dd44328f562313385732d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:11.940691 7982 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69a4acdbe7dd44328f562313385732d7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:11.941293 7982 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69a4acdbe7dd44328f562313385732d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a4acdbe7dd44328f562313385732d7" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 44859 } }
I20260810 07:19:11.941419 7982 leader_election.cc:302] T 00000000000000000000000000000000 P 69a4acdbe7dd44328f562313385732d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69a4acdbe7dd44328f562313385732d7; no voters:
I20260810 07:19:11.941668 7982 leader_election.cc:288] T 00000000000000000000000000000000 P 69a4acdbe7dd44328f562313385732d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:11.941684 7987 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69a4acdbe7dd44328f562313385732d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:11.941922 7987 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69a4acdbe7dd44328f562313385732d7 [term 1 LEADER]: Becoming Leader. State: Replica: 69a4acdbe7dd44328f562313385732d7, State: Running, Role: LEADER
I20260810 07:19:11.941968 7982 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69a4acdbe7dd44328f562313385732d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:11.942066 7987 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69a4acdbe7dd44328f562313385732d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a4acdbe7dd44328f562313385732d7" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 44859 } }
I20260810 07:19:11.942907 7989 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69a4acdbe7dd44328f562313385732d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69a4acdbe7dd44328f562313385732d7. Latest consensus state: current_term: 1 leader_uuid: "69a4acdbe7dd44328f562313385732d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a4acdbe7dd44328f562313385732d7" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 44859 } } }
I20260810 07:19:11.943131 7986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:11.937421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52953 (local address 127.15.217.126:44859)
0810 07:19:11.937598 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:11.937607 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:11.937653 (+ 46us) server_negotiation.cc:407] Connection header received
0810 07:19:11.938128 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:11.938137 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:11.938223 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:11.938393 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:11.938649 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:11.939578 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:11.940489 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:11.941098 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:11.941359 (+ 261us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:11.942247 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:11.942253 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:11.942253 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:11.942472 (+ 219us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:11.942518 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:11.942536 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:11.942613 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:11.942769 (+ 156us) server_negotiation.cc:299] Negotiation successful
0810 07:19:11.942846 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":84,"threads_started":1}
I20260810 07:19:11.943101 7988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69a4acdbe7dd44328f562313385732d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69a4acdbe7dd44328f562313385732d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a4acdbe7dd44328f562313385732d7" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 44859 } } }
I20260810 07:19:11.943208 7988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69a4acdbe7dd44328f562313385732d7 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:11.943114 7989 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69a4acdbe7dd44328f562313385732d7 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:11.943495 7997 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:11.943992 7997 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:11.944424 16229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:19:11.945626 7997 catalog_manager.cc:1347] Generated new cluster ID: d68420d8c3364e65b76db9c49448f46a
I20260810 07:19:11.945681 7997 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:11.950268 7997 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:11.950766 7997 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:11.954658 7986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:11.946096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55655 (local address 127.15.217.126:44859)
0810 07:19:11.946138 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:11.946140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:11.948699 (+ 2559us) server_negotiation.cc:407] Connection header received
0810 07:19:11.948770 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:11.948771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:11.948783 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:11.948849 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:11.949058 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:11.949762 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:11.953138 (+ 3376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:11.953468 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:11.953562 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:11.954326 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:11.954327 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:11.954328 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:11.954333 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:11.954353 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:11.954358 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:11.954414 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:11.954526 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 07:19:11.954574 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 07:19:11.956132 7997 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69a4acdbe7dd44328f562313385732d7: Generated new TSK 0
I20260810 07:19:11.956317 7997 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:19:11.960330 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 7906
2026-08-10T07:19:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1409 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:12 dist-test-slave-10pc krb5kdc[8015](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:12 dist-test-slave-10pc krb5kdc[8015](info): set up 2 sockets
Aug 10 07:19:12 dist-test-slave-10pc krb5kdc[8015](info): commencing operation
krb5kdc: starting...
W20260810 07:19:13.113097 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:19:13 dist-test-slave-10pc krb5kdc[8015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
I20260810 07:19:13.256963 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:40223
--webserver_interface=127.15.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:41715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:40223
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:13.348780 8030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:13.349020 8030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:13.349046 8030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:13.350212 8030 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:13.350255 8030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:13.350270 8030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:13.350288 8030 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:13.350301 8030 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:13.351812 8030 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:41715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:40223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:40223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:13.352066 8030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:13.352267 8030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:13.357473 8039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:13.357548 8037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:13.357654 8030 server_base.cc:1034] running on GCE node
W20260810 07:19:13.357478 8036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:13.357955 8030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:13.358301 8030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:13.359496 8030 hybrid_clock.cc:648] HybridClock initialized: now 1786346353359470 us; error 56 us; skew 500 ppm
Aug 10 07:19:13 dist-test-slave-10pc krb5kdc[8015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346353, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:13.362243 8030 init.cc:377] Logged in from keytab as kudu/127.15.217.126@KRBTEST.COM (short username kudu)
I20260810 07:19:13.362684 8030 webserver.cc:460] Webserver started at http://127.15.217.126:41569/ using document root <none> and password file <none>
I20260810 07:19:13.362879 8030 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:13.362923 8030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:13.363018 8030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:13.364131 8030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "cb62cfa97379427b9f6916f59e8a95db"
format_stamp: "Formatted at 2026-08-10 07:19:13 on dist-test-slave-10pc"
server_key: "3145b2daa59079f8a7a85051eba6246a"
server_key_iv: "0a9e434897f5b4e3b9fc9dff027f77d8"
server_key_version: "encryptionkey@0"
I20260810 07:19:13.364609 8030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "cb62cfa97379427b9f6916f59e8a95db"
format_stamp: "Formatted at 2026-08-10 07:19:13 on dist-test-slave-10pc"
server_key: "3145b2daa59079f8a7a85051eba6246a"
server_key_iv: "0a9e434897f5b4e3b9fc9dff027f77d8"
server_key_version: "encryptionkey@0"
I20260810 07:19:13.366024 8030 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:19:13.366767 8046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:13.366947 8030 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 07:19:13.367043 8030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "cb62cfa97379427b9f6916f59e8a95db"
format_stamp: "Formatted at 2026-08-10 07:19:13 on dist-test-slave-10pc"
server_key: "3145b2daa59079f8a7a85051eba6246a"
server_key_iv: "0a9e434897f5b4e3b9fc9dff027f77d8"
server_key_version: "encryptionkey@0"
I20260810 07:19:13.367132 8030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:13.370601 8030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:13.372776 8030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:13.372941 8030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:13.377038 8030 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:40223
I20260810 07:19:13.377138 8102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:40223 every 8 connection(s)
I20260810 07:19:13.377453 8030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:13.378532 8103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:13.380928 8103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb62cfa97379427b9f6916f59e8a95db: Bootstrap starting.
I20260810 07:19:13.381570 8103 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb62cfa97379427b9f6916f59e8a95db: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:13.381891 8103 log.cc:824] T 00000000000000000000000000000000 P cb62cfa97379427b9f6916f59e8a95db: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:13.382628 8103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb62cfa97379427b9f6916f59e8a95db: No bootstrap required, opened a new log
I20260810 07:19:13.383705 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 8030
I20260810 07:19:13.383810 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:19:13.384027 16229 external_mini_cluster.cc:1428] Setting key 1b6f98f08fba53d28d827a7bc18c0e40
I20260810 07:19:13.384898 8103 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb62cfa97379427b9f6916f59e8a95db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb62cfa97379427b9f6916f59e8a95db" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 40223 } }
I20260810 07:19:13.385106 8103 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb62cfa97379427b9f6916f59e8a95db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:13.385145 8103 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb62cfa97379427b9f6916f59e8a95db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb62cfa97379427b9f6916f59e8a95db, State: Initialized, Role: FOLLOWER
I20260810 07:19:13.385311 8103 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb62cfa97379427b9f6916f59e8a95db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb62cfa97379427b9f6916f59e8a95db" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 40223 } }
I20260810 07:19:13.385417 8103 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb62cfa97379427b9f6916f59e8a95db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:13.385532 8103 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb62cfa97379427b9f6916f59e8a95db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:13.385592 8103 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb62cfa97379427b9f6916f59e8a95db [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:13.386595 8107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:13.384638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49561 (local address 127.15.217.126:40223)
0810 07:19:13.384882 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:13.384892 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:13.384940 (+ 48us) server_negotiation.cc:407] Connection header received
0810 07:19:13.385066 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:13.385074 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:13.385106 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:13.385260 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:13.386075 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:13.386080 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:13.386081 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:13.386087 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:13.386150 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:13.386167 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:13.386257 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:13.386311 (+ 54us) server_negotiation.cc:299] Negotiation successful
0810 07:19:13.386356 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":123,"threads_started":1}
I20260810 07:19:13.386762 8103 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb62cfa97379427b9f6916f59e8a95db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb62cfa97379427b9f6916f59e8a95db" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 40223 } }
I20260810 07:19:13.386895 8103 leader_election.cc:302] T 00000000000000000000000000000000 P cb62cfa97379427b9f6916f59e8a95db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb62cfa97379427b9f6916f59e8a95db; no voters:
I20260810 07:19:13.387101 8103 leader_election.cc:288] T 00000000000000000000000000000000 P cb62cfa97379427b9f6916f59e8a95db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:13.387133 8108 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb62cfa97379427b9f6916f59e8a95db [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:13.387252 8108 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb62cfa97379427b9f6916f59e8a95db [term 1 LEADER]: Becoming Leader. State: Replica: cb62cfa97379427b9f6916f59e8a95db, State: Running, Role: LEADER
I20260810 07:19:13.387394 8103 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb62cfa97379427b9f6916f59e8a95db [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:13.387394 8108 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb62cfa97379427b9f6916f59e8a95db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb62cfa97379427b9f6916f59e8a95db" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 40223 } }
I20260810 07:19:13.387689 16229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:19:13.389451 8110 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb62cfa97379427b9f6916f59e8a95db [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb62cfa97379427b9f6916f59e8a95db. Latest consensus state: current_term: 1 leader_uuid: "cb62cfa97379427b9f6916f59e8a95db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb62cfa97379427b9f6916f59e8a95db" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 40223 } } }
I20260810 07:19:13.389761 8110 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb62cfa97379427b9f6916f59e8a95db [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:13.389680 8109 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb62cfa97379427b9f6916f59e8a95db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb62cfa97379427b9f6916f59e8a95db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb62cfa97379427b9f6916f59e8a95db" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 40223 } } }
I20260810 07:19:13.390110 8109 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb62cfa97379427b9f6916f59e8a95db [sys.catalog]: This master's current role is: LEADER
W20260810 07:19:13.389621 8128 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cb62cfa97379427b9f6916f59e8a95db: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:19:13.390509 8128 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:19:13.390600 8133 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:13.390801 8133 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:13.390949 8107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:13.389390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43527 (local address 127.15.217.126:40223)
0810 07:19:13.389432 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:13.389434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:13.389796 (+ 362us) server_negotiation.cc:407] Connection header received
0810 07:19:13.389872 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:13.389873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:13.389879 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:13.389943 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:13.390644 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:13.390646 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:13.390646 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:13.390659 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:13.390677 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:13.390682 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:13.390714 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:13.390843 (+ 129us) server_negotiation.cc:299] Negotiation successful
0810 07:19:13.390858 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 07:19:13.392697 8133 catalog_manager.cc:1347] Generated new cluster ID: bba4bdeadc20449f9cd54f7842c5cfe0
I20260810 07:19:13.392756 8133 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:13.406654 8133 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:13.406867 8133 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:13.411945 8133 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb62cfa97379427b9f6916f59e8a95db: Generated new TSK 0
I20260810 07:19:13.412184 8133 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:19:13.414682 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 8030
2026-08-10T07:19:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1452 ms)
[----------] 18 tests from AuthTokenIssuingTest (25510 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:13 dist-test-slave-10pc krb5kdc[8138](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:13 dist-test-slave-10pc krb5kdc[8138](info): set up 2 sockets
Aug 10 07:19:13 dist-test-slave-10pc krb5kdc[8138](info): commencing operation
krb5kdc: starting...
W20260810 07:19:14.527643 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:19:14 dist-test-slave-10pc krb5kdc[8138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
I20260810 07:19:14.663743 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:36909
--webserver_interface=127.15.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:44889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:36909,127.15.217.125:37137,127.15.217.124:39909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:14.751588 8152 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 07:19:14.751847 8152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:14.751864 8152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:14.751888 8152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:14.753213 8152 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:14.753268 8152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:14.753283 8152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:14.753295 8152 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:14.753309 8152 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:14.754899 8152 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:44889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:36909,127.15.217.125:37137,127.15.217.124:39909
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:36909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:14.755194 8152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:14.755455 8152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:14.760710 8162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:14.760696 8158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:14.760699 8159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:14.760731 8152 server_base.cc:1034] running on GCE node
I20260810 07:19:14.761204 8152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:14.761473 8152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:14.762643 8152 hybrid_clock.cc:648] HybridClock initialized: now 1786346354762629 us; error 46 us; skew 500 ppm
Aug 10 07:19:14 dist-test-slave-10pc krb5kdc[8138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346354, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:14.765451 8152 init.cc:377] Logged in from keytab as kudu/127.15.217.126@KRBTEST.COM (short username kudu)
I20260810 07:19:14.765949 8152 webserver.cc:460] Webserver started at http://127.15.217.126:45285/ using document root <none> and password file <none>
I20260810 07:19:14.766211 8152 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:14.766259 8152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:14.766433 8152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:14.767575 8152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "b5e76b7b22184ec2891dffbb4cbecc8b"
format_stamp: "Formatted at 2026-08-10 07:19:14 on dist-test-slave-10pc"
server_key: "b167a5756b77e3cefec32965b64b22a7"
server_key_iv: "ce9a01aa10ec002104cc962ad80972af"
server_key_version: "encryptionkey@0"
I20260810 07:19:14.767980 8152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "b5e76b7b22184ec2891dffbb4cbecc8b"
format_stamp: "Formatted at 2026-08-10 07:19:14 on dist-test-slave-10pc"
server_key: "b167a5756b77e3cefec32965b64b22a7"
server_key_iv: "ce9a01aa10ec002104cc962ad80972af"
server_key_version: "encryptionkey@0"
I20260810 07:19:14.769593 8152 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:19:14.770396 8169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:14.770577 8152 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:14.770691 8152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "b5e76b7b22184ec2891dffbb4cbecc8b"
format_stamp: "Formatted at 2026-08-10 07:19:14 on dist-test-slave-10pc"
server_key: "b167a5756b77e3cefec32965b64b22a7"
server_key_iv: "ce9a01aa10ec002104cc962ad80972af"
server_key_version: "encryptionkey@0"
I20260810 07:19:14.770773 8152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:14.784744 8152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:14.787297 8152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:14.787485 8152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:14.792064 8152 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:36909
I20260810 07:19:14.792104 8225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:36909 every 8 connection(s)
I20260810 07:19:14.792551 8152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:14.793473 8226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:14.795075 8226 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.217.126" port: 36909 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:19:14.800251 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 8152
I20260810 07:19:14.800347 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:19:14.800563 16229 external_mini_cluster.cc:1428] Setting key 9b4d8f5f415dc9e4d4e9034f9c61088d
Aug 10 07:19:14 dist-test-slave-10pc krb5kdc[8138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346354, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:19:14.806470 8229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:14.795960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37590 (local address 127.15.217.126:36909)
0810 07:19:14.796096 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:14.796103 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:14.796140 (+ 37us) server_negotiation.cc:407] Connection header received
0810 07:19:14.796667 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:14.796669 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:14.796687 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:14.796762 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:14.797324 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:14.798166 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:14.798994 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:14.799301 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:14.799382 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:14.801711 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:14.801719 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:14.801720 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:14.801728 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:14.804817 (+ 3089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:14.805310 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:14.805312 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:14.805312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:14.805392 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:14.805667 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:14.805667 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:14.805668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:14.805883 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:14.805946 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:14.806189 (+ 243us) server_negotiation.cc:299] Negotiation successful
0810 07:19:14.806288 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":73,"threads_started":1}
I20260810 07:19:14.806599 8228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:14.795799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:36909 (local address 127.0.0.1:37590)
0810 07:19:14.795960 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:14.796000 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:14.796517 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:14.796793 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:14.796796 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:14.797050 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:14.797241 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:14.797245 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:14.798249 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:14.798250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:14.798923 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:14.798924 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:14.799381 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:14.799382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:14.799440 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:14.800042 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:14.800051 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:14.801625 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:14.804900 (+ 3275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:14.804908 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:14.804910 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:14.805222 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:14.805462 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:14.805463 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:14.805463 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:14.805588 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:14.805954 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:14.805962 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:14.806042 (+ 80us) client_negotiation.cc:769] Sending connection context
0810 07:19:14.806160 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 07:19:14.806389 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":10,"thread_start_us":71,"threads_started":1}
I20260810 07:19:14.807435 8226 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.217.125" port: 37137 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:19:14.807843 8228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:14.807711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:37137 (local address 127.0.0.1:35668)
0810 07:19:14.807769 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:14.807818 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.125:37137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260810 07:19:14.808034 8171 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.217.125:37137: connect: Connection refused (error 111)
I20260810 07:19:14.808542 8228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:14.808404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:37137 (local address 127.0.0.1:35672)
0810 07:19:14.808489 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:14.808519 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.125:37137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260810 07:19:14.808708 8226 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.217.125:37137: Network error: Client connection negotiation failed: client connection to 127.15.217.125:37137: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.15.217.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.125.keytab.
Entry for principal kudu/127.15.217.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.125.keytab.
I20260810 07:19:14.831918 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.125:37137
--webserver_interface=127.15.217.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.125.keytab
--principal=kudu/127.15.217.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:44889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:36909,127.15.217.125:37137,127.15.217.124:39909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:19:14.857951 8226 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.217.125" port: 37137 } attempt: 1
I20260810 07:19:14.858464 8228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:14.858325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:37137 (local address 127.0.0.1:35682)
0810 07:19:14.858390 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:14.858431 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.125:37137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 07:19:14.858832 8228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:14.858735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:37137 (local address 127.0.0.1:35694)
0810 07:19:14.858781 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:14.858807 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.125:37137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260810 07:19:14.858923 8226 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.217.125:37137: Network error: Client connection negotiation failed: client connection to 127.15.217.125:37137: connect: Connection refused (error 111)
W20260810 07:19:14.918589 8234 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 07:19:14.918803 8234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:14.918821 8234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:14.918841 8234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:14.920024 8234 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:14.920064 8234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:14.920079 8234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:14.920091 8234 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:14.920149 8234 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:14.921659 8234 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:44889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:36909,127.15.217.125:37137,127.15.217.124:39909
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.125.keytab
--principal=kudu/127.15.217.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.125:37137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.217.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:14.921924 8234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:14.922163 8234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:19:14.927114 8226 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.217.125" port: 37137 } attempt: 2
W20260810 07:19:14.927162 8239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:14.927186 8242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:14.927316 8240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:14.927663 8228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:14.927503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:37137 (local address 127.0.0.1:35702)
0810 07:19:14.927580 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:14.927631 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.125:37137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260810 07:19:14.927634 8234 server_base.cc:1034] running on GCE node
I20260810 07:19:14.927959 8234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:14.928095 8228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:14.927965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:37137 (local address 127.0.0.1:35712)
0810 07:19:14.928022 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:14.928069 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.125:37137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 07:19:14.928222 8226 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.217.125:37137: Network error: Client connection negotiation failed: client connection to 127.15.217.125:37137: connect: Connection refused (error 111)
I20260810 07:19:14.928349 8234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:14.929514 8234 hybrid_clock.cc:648] HybridClock initialized: now 1786346354929503 us; error 27 us; skew 500 ppm
Aug 10 07:19:14 dist-test-slave-10pc krb5kdc[8138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346354, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:14.932168 8234 init.cc:377] Logged in from keytab as kudu/127.15.217.125@KRBTEST.COM (short username kudu)
I20260810 07:19:14.932718 8234 webserver.cc:460] Webserver started at http://127.15.217.125:45319/ using document root <none> and password file <none>
I20260810 07:19:14.932958 8234 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:14.933029 8234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:14.933177 8234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:14.934309 8234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-1/data/instance:
uuid: "53402574c2de41a4ae5a87b900d040bf"
format_stamp: "Formatted at 2026-08-10 07:19:14 on dist-test-slave-10pc"
server_key: "7d043b7dac1368d15a1f5c2d28c04821"
server_key_iv: "55dd84c70756383e31b08c02622771ff"
server_key_version: "encryptionkey@0"
I20260810 07:19:14.934703 8234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-1/wal/instance:
uuid: "53402574c2de41a4ae5a87b900d040bf"
format_stamp: "Formatted at 2026-08-10 07:19:14 on dist-test-slave-10pc"
server_key: "7d043b7dac1368d15a1f5c2d28c04821"
server_key_iv: "55dd84c70756383e31b08c02622771ff"
server_key_version: "encryptionkey@0"
I20260810 07:19:14.936146 8234 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:19:14.936995 8250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:14.937181 8234 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 07:19:14.937291 8234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-1/wal
uuid: "53402574c2de41a4ae5a87b900d040bf"
format_stamp: "Formatted at 2026-08-10 07:19:14 on dist-test-slave-10pc"
server_key: "7d043b7dac1368d15a1f5c2d28c04821"
server_key_iv: "55dd84c70756383e31b08c02622771ff"
server_key_version: "encryptionkey@0"
I20260810 07:19:14.937369 8234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:14.944950 8234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:14.946823 8234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:14.946990 8234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:14.951455 8234 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.125:37137
I20260810 07:19:14.951490 8309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.125:37137 every 8 connection(s)
I20260810 07:19:14.951875 8234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-1/data/info.pb
I20260810 07:19:14.952841 8310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:14.954394 8310 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.217.126" port: 36909 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:19:14.957968 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 8234
I20260810 07:19:14.958202 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-1/wal/instance
I20260810 07:19:14.958427 16229 external_mini_cluster.cc:1428] Setting key 572e1157863942fb7035760702ea620b
Aug 10 07:19:14 dist-test-slave-10pc krb5kdc[8138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346354, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.125@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:19:14.963918 8229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:14.955111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37596 (local address 127.15.217.126:36909)
0810 07:19:14.955179 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:14.955182 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:14.955394 (+ 212us) server_negotiation.cc:407] Connection header received
0810 07:19:14.955870 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:14.955871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:14.955882 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:14.955941 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:14.956661 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:14.957182 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:14.958197 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:14.958477 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:14.958561 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:14.960939 (+ 2378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:14.960941 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:14.960941 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:14.960949 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:14.962443 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:14.962876 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:14.962877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:14.962878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:14.962944 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:14.963230 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:14.963231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:14.963232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:14.963361 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:14.963437 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:14.963731 (+ 294us) server_negotiation.cc:299] Negotiation successful
0810 07:19:14.963820 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 07:19:14.964141 8312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:14.955077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:36909 (local address 127.0.0.1:37596)
0810 07:19:14.955259 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:14.955304 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:14.955785 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:14.956027 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:14.956030 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:14.956303 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:14.956574 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:14.956580 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:14.957273 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:14.957274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:14.958071 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:14.958073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:14.958594 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:14.958595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:14.958726 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:14.959332 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:14.959351 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:14.960823 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:14.962536 (+ 1713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:14.962545 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:14.962548 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:14.962794 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:14.963016 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:14.963017 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:14.963018 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:14.963142 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:14.963430 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:14.963438 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:14.963591 (+ 153us) client_negotiation.cc:769] Sending connection context
0810 07:19:14.963708 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 07:19:14.963881 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":105,"threads_started":1}
I20260810 07:19:14.964808 8310 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.217.125" port: 37137 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.15.217.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.124@KRBTEST.COM" created.
Aug 10 07:19:14 dist-test-slave-10pc krb5kdc[8138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346354, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.125@KRBTEST.COM for kudu/127.15.217.125@KRBTEST.COM
I20260810 07:19:14.973896 8312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:14.965119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:37137 (local address 127.0.0.1:35714)
0810 07:19:14.965191 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:14.965202 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:14.965250 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:14.965488 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:14.965489 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:14.965634 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:14.965686 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:14.965688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:14.966545 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:14.966546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:14.967047 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:14.967048 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:14.967568 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:14.967569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:14.967635 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:14.968046 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:14.968051 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:14.970502 (+ 2451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:14.972683 (+ 2181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:14.972684 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:14.972684 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:14.972849 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:14.973093 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:14.973094 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:14.973094 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:14.973175 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:14.973469 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:14.973470 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:14.973526 (+ 56us) client_negotiation.cc:769] Sending connection context
0810 07:19:14.973605 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 07:19:14.973726 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 07:19:14.973965 8314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:14.965173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35714 (local address 127.15.217.125:37137)
0810 07:19:14.965342 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:14.965346 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:14.965369 (+ 23us) server_negotiation.cc:407] Connection header received
0810 07:19:14.965394 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:14.965396 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:14.965407 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:14.965487 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:14.965770 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:14.966456 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:14.967133 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:14.967490 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:14.967571 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:14.970615 (+ 3044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:14.970625 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:14.970626 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:14.970633 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:14.972581 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:14.972931 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:14.972935 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:14.972935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:14.973004 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:14.973245 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:14.973246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:14.973246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:14.973397 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:14.973464 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:14.973629 (+ 165us) server_negotiation.cc:299] Negotiation successful
0810 07:19:14.973832 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":109,"server-negotiator.queue_time_us":140,"thread_start_us":89,"threads_started":1}
I20260810 07:19:14.974589 8310 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.217.124" port: 39909 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:19:14.974974 8312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:14.974865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:39909 (local address 127.0.0.1:41308)
0810 07:19:14.974899 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:14.974946 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.124:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260810 07:19:14.975144 8251 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.217.124:39909: connect: Connection refused (error 111)
I20260810 07:19:14.975579 8312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:14.975488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:39909 (local address 127.0.0.1:41324)
0810 07:19:14.975534 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:14.975558 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.124:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260810 07:19:14.975728 8310 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.217.124:39909: Network error: Client connection negotiation failed: client connection to 127.15.217.124:39909: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.124.keytab.
Entry for principal kudu/127.15.217.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.124.keytab.
I20260810 07:19:14.987190 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.124:39909
--webserver_interface=127.15.217.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.124.keytab
--principal=kudu/127.15.217.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:44889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:36909,127.15.217.125:37137,127.15.217.124:39909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:19:15.019382 8226 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.217.125" port: 37137 } attempt: 3
Aug 10 07:19:15 dist-test-slave-10pc krb5kdc[8138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346354, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for kudu/127.15.217.125@KRBTEST.COM
I20260810 07:19:15.024988 8310 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.217.124" port: 39909 } attempt: 1
I20260810 07:19:15.025467 8312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:15.025304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:39909 (local address 127.0.0.1:41332)
0810 07:19:15.025375 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:15.025421 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.124:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 07:19:15.025887 8312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:15.025757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:39909 (local address 127.0.0.1:41344)
0810 07:19:15.025815 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:15.025853 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.124:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260810 07:19:15.026214 8310 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.217.124:39909: Network error: Client connection negotiation failed: client connection to 127.15.217.124:39909: connect: Connection refused (error 111)
I20260810 07:19:15.027045 8314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:15.019897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35728 (local address 127.15.217.125:37137)
0810 07:19:15.019980 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:15.019982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:15.019995 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:19:15.020039 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:15.020040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:15.020057 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:15.020119 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:15.020500 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.021006 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.021798 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.022037 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.022119 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:15.024090 (+ 1971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:15.024091 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:15.024092 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:15.024107 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:15.025848 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:15.026215 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:15.026216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:15.026216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:15.026304 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:15.026577 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:15.026578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:15.026578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:15.026687 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:15.026736 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:15.026930 (+ 194us) server_negotiation.cc:299] Negotiation successful
0810 07:19:15.026992 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 07:19:15.027184 8228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:15.019811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:37137 (local address 127.0.0.1:35728)
0810 07:19:15.019880 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:15.019897 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:15.019969 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:15.020130 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:15.020131 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:15.020309 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:15.020380 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:15.020383 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.021095 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.021096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:15.021706 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:15.021708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.022117 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.022118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:15.022206 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:15.022742 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:15.022746 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:15.023991 (+ 1245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:15.025928 (+ 1937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:15.025929 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:15.025930 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:15.026127 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:15.026399 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:15.026400 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:15.026401 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:15.026497 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:15.026756 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:15.026757 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:15.026820 (+ 63us) client_negotiation.cc:769] Sending connection context
0810 07:19:15.026916 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 07:19:15.027048 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 07:19:15.027638 8226 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.217.124" port: 39909 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:19:15.028025 8228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:15.027911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:39909 (local address 127.0.0.1:41358)
0810 07:19:15.027963 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:15.027997 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.124:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260810 07:19:15.028375 8228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:15.028278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:39909 (local address 127.0.0.1:41372)
0810 07:19:15.028329 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:15.028356 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.124:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260810 07:19:15.028497 8226 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.217.124:39909: Network error: Client connection negotiation failed: client connection to 127.15.217.124:39909: connect: Connection refused (error 111)
I20260810 07:19:15.059681 8226 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.217.124" port: 39909 } attempt: 1
I20260810 07:19:15.060300 8228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:15.060125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:39909 (local address 127.0.0.1:41384)
0810 07:19:15.060215 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:15.060266 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.124:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 07:19:15.060791 8228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:15.060671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:39909 (local address 127.0.0.1:41394)
0810 07:19:15.060736 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:15.060767 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.124:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 07:19:15.060935 8226 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.217.124:39909: Network error: Client connection negotiation failed: client connection to 127.15.217.124:39909: connect: Connection refused (error 111)
W20260810 07:19:15.075913 8317 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 07:19:15.076246 8317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:15.076294 8317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:15.076340 8317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:15.077641 8317 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:15.077711 8317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:15.077735 8317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:15.077756 8317 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:15.077786 8317 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:15.079336 8317 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:44889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:36909,127.15.217.125:37137,127.15.217.124:39909
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.124.keytab
--principal=kudu/127.15.217.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.124:39909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.217.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:15.079646 8317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:15.079898 8317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:15.085031 8323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:15.085057 8322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:15.085062 8325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:15.085309 8317 server_base.cc:1034] running on GCE node
I20260810 07:19:15.085567 8317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:15.085886 8317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:15.087073 8317 hybrid_clock.cc:648] HybridClock initialized: now 1786346355087059 us; error 45 us; skew 500 ppm
Aug 10 07:19:15 dist-test-slave-10pc krb5kdc[8138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346355, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:15.089797 8317 init.cc:377] Logged in from keytab as kudu/127.15.217.124@KRBTEST.COM (short username kudu)
I20260810 07:19:15.090212 8317 webserver.cc:460] Webserver started at http://127.15.217.124:38983/ using document root <none> and password file <none>
I20260810 07:19:15.090402 8317 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:15.090442 8317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:15.090534 8317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:15.091599 8317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-2/data/instance:
uuid: "33d8bfb0279c474ead72ac856b4ed5a9"
format_stamp: "Formatted at 2026-08-10 07:19:15 on dist-test-slave-10pc"
server_key: "82737d50f5f867d35a650f250ae06f42"
server_key_iv: "5923d14955b0e412c692139f37821943"
server_key_version: "encryptionkey@0"
I20260810 07:19:15.091950 8317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-2/wal/instance:
uuid: "33d8bfb0279c474ead72ac856b4ed5a9"
format_stamp: "Formatted at 2026-08-10 07:19:15 on dist-test-slave-10pc"
server_key: "82737d50f5f867d35a650f250ae06f42"
server_key_iv: "5923d14955b0e412c692139f37821943"
server_key_version: "encryptionkey@0"
I20260810 07:19:15.093441 8317 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:19:15.094183 8334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:15.094358 8317 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 07:19:15.094363 8310 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.217.124" port: 39909 } attempt: 2
I20260810 07:19:15.094472 8317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-2/wal
uuid: "33d8bfb0279c474ead72ac856b4ed5a9"
format_stamp: "Formatted at 2026-08-10 07:19:15 on dist-test-slave-10pc"
server_key: "82737d50f5f867d35a650f250ae06f42"
server_key_iv: "5923d14955b0e412c692139f37821943"
server_key_version: "encryptionkey@0"
I20260810 07:19:15.094554 8317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:15.094767 8312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:15.094643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:39909 (local address 127.0.0.1:41400)
0810 07:19:15.094705 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:15.094734 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.124:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 07:19:15.095113 8312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:15.095012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:39909 (local address 127.0.0.1:41410)
0810 07:19:15.095062 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:15.095089 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.124:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260810 07:19:15.095221 8310 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.217.124:39909: Network error: Client connection negotiation failed: client connection to 127.15.217.124:39909: connect: Connection refused (error 111)
I20260810 07:19:15.106508 8317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:15.108950 8317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:15.109144 8317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:15.113189 8317 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.124:39909
I20260810 07:19:15.113245 8390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.124:39909 every 8 connection(s)
I20260810 07:19:15.113623 8317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-2/data/info.pb
I20260810 07:19:15.114553 8391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:15.116092 8391 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.217.126" port: 36909 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:19:15 dist-test-slave-10pc krb5kdc[8138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346355, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.124@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:19:15.123217 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 8317
I20260810 07:19:15.123355 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346081759552-16229-0/minicluster-data/master-2/wal/instance
I20260810 07:19:15.123561 16229 external_mini_cluster.cc:1428] Setting key a859577adfd24df9704f250f20ca4568
I20260810 07:19:15.125430 8229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:15.116801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37608 (local address 127.15.217.126:36909)
0810 07:19:15.116882 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:15.116884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:15.117063 (+ 179us) server_negotiation.cc:407] Connection header received
0810 07:19:15.117599 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:15.117600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:15.117613 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:15.117659 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:15.118291 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.118784 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.119793 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.119987 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.120076 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:15.122228 (+ 2152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:15.122230 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:15.122231 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:15.122239 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:15.123774 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:15.124277 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:15.124279 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:15.124279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:15.124352 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:15.124696 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:15.124697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:15.124698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:15.124815 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:15.124867 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:15.125242 (+ 375us) server_negotiation.cc:299] Negotiation successful
0810 07:19:15.125334 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 07:19:15.125638 8394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:15.116783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:36909 (local address 127.0.0.1:37608)
0810 07:19:15.116942 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:15.116979 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:15.117480 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:15.117750 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:15.117753 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:15.117996 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:15.118192 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:15.118197 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.118885 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.118886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:15.119694 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:15.119696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.120088 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.120089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:15.120217 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:15.120763 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:15.120772 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:15.122123 (+ 1351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:15.123869 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:15.123878 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:15.123880 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:15.124188 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:15.124435 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:15.124436 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:15.124437 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:15.124612 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:15.124885 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:15.124894 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:15.125084 (+ 190us) client_negotiation.cc:769] Sending connection context
0810 07:19:15.125206 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 07:19:15.125360 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":68,"threads_started":1}
I20260810 07:19:15.126207 8391 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.217.125" port: 37137 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:19:15 dist-test-slave-10pc krb5kdc[8138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.124@KRBTEST.COM
Aug 10 07:19:15 dist-test-slave-10pc krb5kdc[8138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346355, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.124@KRBTEST.COM for kudu/127.15.217.125@KRBTEST.COM
I20260810 07:19:15.135737 8314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:15.126678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35732 (local address 127.15.217.125:37137)
0810 07:19:15.126745 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:15.126747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:15.126830 (+ 83us) server_negotiation.cc:407] Connection header received
0810 07:19:15.126918 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:15.126919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:15.126930 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:15.126959 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:15.127320 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.128031 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.128975 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.129485 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.129584 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:15.132156 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:15.132157 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:15.132157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:15.132167 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:15.134390 (+ 2223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:15.134797 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:15.134798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:15.134798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:15.134862 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:15.135092 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:15.135093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:15.135093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:15.135257 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:15.135305 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:15.135531 (+ 226us) server_negotiation.cc:299] Negotiation successful
0810 07:19:15.135615 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 07:19:15.136307 8394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:15.126479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:37137 (local address 127.0.0.1:35732)
0810 07:19:15.126777 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:15.126787 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:15.126851 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:15.127017 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:15.127018 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:15.127170 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:15.127228 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:15.127230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.128127 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.128128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:15.128858 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:15.128861 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.129666 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.129667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:15.129783 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:15.130327 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:15.130332 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:15.132052 (+ 1720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:15.134490 (+ 2438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:15.134491 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:15.134492 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:15.134700 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:15.134948 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:15.134949 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:15.134949 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:15.135012 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:15.135356 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:15.135357 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:15.135405 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 07:19:15.136065 (+ 660us) client_negotiation.cc:240] Negotiation successful
0810 07:19:15.136190 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282}
I20260810 07:19:15.137585 8226 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.217.124" port: 39909 } attempt: 2
I20260810 07:19:15.137737 8395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:15.124001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41412 (local address 127.15.217.124:39909)
0810 07:19:15.124211 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:15.124215 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:15.124283 (+ 68us) server_negotiation.cc:407] Connection header received
0810 07:19:15.124339 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:15.124342 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:15.124357 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:15.124433 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:15.125226 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.126237 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.127294 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.127738 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.127811 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:15.132184 (+ 4373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:15.132197 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:15.132198 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:15.132204 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:15.135203 (+ 2999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:15.135876 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:15.135882 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:15.135883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:15.135975 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:15.136263 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:15.136264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:15.136264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:15.136568 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:15.137468 (+ 900us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:15.137523 (+ 55us) server_negotiation.cc:299] Negotiation successful
0810 07:19:15.137613 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":95,"threads_started":1}
I20260810 07:19:15.137768 8391 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.217.124" port: 39909 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:19:15 dist-test-slave-10pc krb5kdc[8138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346354, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for kudu/127.15.217.124@KRBTEST.COM
Aug 10 07:19:15 dist-test-slave-10pc krb5kdc[8138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346355, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.124@KRBTEST.COM for kudu/127.15.217.124@KRBTEST.COM
Aug 10 07:19:15 dist-test-slave-10pc krb5kdc[8138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:19:15.147426 8228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:15.137878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:39909 (local address 127.0.0.1:41420)
0810 07:19:15.137966 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:15.137977 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:15.138019 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:15.138201 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:15.138201 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:15.138381 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:15.138440 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:15.138444 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.139383 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.139384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:15.140664 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:15.140667 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.141137 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.141139 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:15.141201 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:15.141610 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:15.141614 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:15.144049 (+ 2435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:15.146204 (+ 2155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:15.146205 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:15.146205 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:15.146423 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:15.146687 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:15.146689 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:15.146689 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:15.146760 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:15.147071 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:15.147072 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:15.147120 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 07:19:15.147213 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 07:19:15.147319 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260810 07:19:15.147430 8394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:15.138009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:39909 (local address 127.0.0.1:41436)
0810 07:19:15.138048 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:15.138056 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:15.138088 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:15.138592 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:15.138592 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:15.138733 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:15.138762 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:15.138763 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.139611 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.139612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:15.140424 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:15.140426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.140922 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.140923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:15.140986 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:15.141645 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:15.141651 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:15.143516 (+ 1865us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:15.146298 (+ 2782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:15.146299 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:15.146300 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:15.146504 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:15.146720 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:15.146720 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:15.146721 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:15.146779 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:15.147084 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:15.147085 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:15.147123 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 07:19:15.147194 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 07:19:15.147312 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260810 07:19:15.147532 8229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:15.138756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37612 (local address 127.15.217.126:36909)
0810 07:19:15.138797 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:15.138799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:15.138836 (+ 37us) server_negotiation.cc:407] Connection header received
0810 07:19:15.138909 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:15.138910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:15.138919 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:15.138968 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:15.139539 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.140253 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.141355 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.141673 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.141718 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:15.144556 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:15.144557 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:15.144558 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:15.144566 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:15.146070 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:15.146505 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:15.146506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:15.146506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:15.146559 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:15.146781 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:15.146781 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:15.146782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:15.146929 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:15.147118 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:15.147356 (+ 238us) server_negotiation.cc:299] Negotiation successful
0810 07:19:15.147437 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 07:19:15.147607 8397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:15.138383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41436 (local address 127.15.217.124:39909)
0810 07:19:15.138439 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:15.138441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:15.138449 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:19:15.138469 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:15.138469 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:15.138479 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:15.138547 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:15.138834 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.139534 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.140553 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.140842 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.140908 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:15.143603 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:15.143605 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:15.143605 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:15.143612 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:15.146230 (+ 2618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:15.146571 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:15.146573 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:15.146573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:15.146648 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:15.146841 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:15.146841 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:15.146841 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:15.147007 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:15.147062 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:15.147236 (+ 174us) server_negotiation.cc:299] Negotiation successful
0810 07:19:15.147369 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":38}
I20260810 07:19:15.147802 8395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:15.137963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41420 (local address 127.15.217.124:39909)
0810 07:19:15.138093 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:15.138095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:15.138102 (+ 7us) server_negotiation.cc:407] Connection header received
0810 07:19:15.138120 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:15.138121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:15.138131 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:15.138189 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:15.138538 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.139311 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.140752 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.141074 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.141134 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:15.144126 (+ 2992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:15.144127 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:15.144127 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:15.144135 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:15.146083 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:15.146496 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:15.146497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:15.146498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:15.146576 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:15.146828 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:15.146829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:15.146830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:15.146999 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:15.147076 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:15.147644 (+ 568us) server_negotiation.cc:299] Negotiation successful
0810 07:19:15.147725 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":96,"threads_started":1}
I20260810 07:19:15.149329 8391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33d8bfb0279c474ead72ac856b4ed5a9: Bootstrap starting.
I20260810 07:19:15.149506 8226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5e76b7b22184ec2891dffbb4cbecc8b: Bootstrap starting.
I20260810 07:19:15.150022 8391 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33d8bfb0279c474ead72ac856b4ed5a9: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:15.150192 8226 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5e76b7b22184ec2891dffbb4cbecc8b: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:15.150267 8391 log.cc:824] T 00000000000000000000000000000000 P 33d8bfb0279c474ead72ac856b4ed5a9: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:15.150553 8226 log.cc:824] T 00000000000000000000000000000000 P b5e76b7b22184ec2891dffbb4cbecc8b: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:15.151105 8391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33d8bfb0279c474ead72ac856b4ed5a9: No bootstrap required, opened a new log
I20260810 07:19:15.151275 8226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5e76b7b22184ec2891dffbb4cbecc8b: No bootstrap required, opened a new log
I20260810 07:19:15.152937 8391 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33d8bfb0279c474ead72ac856b4ed5a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e76b7b22184ec2891dffbb4cbecc8b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 36909 } } peers { permanent_uuid: "53402574c2de41a4ae5a87b900d040bf" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 37137 } } peers { permanent_uuid: "33d8bfb0279c474ead72ac856b4ed5a9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 39909 } }
I20260810 07:19:15.153117 8391 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33d8bfb0279c474ead72ac856b4ed5a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d8bfb0279c474ead72ac856b4ed5a9, State: Initialized, Role: FOLLOWER
I20260810 07:19:15.153095 8226 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5e76b7b22184ec2891dffbb4cbecc8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e76b7b22184ec2891dffbb4cbecc8b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 36909 } } peers { permanent_uuid: "53402574c2de41a4ae5a87b900d040bf" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 37137 } } peers { permanent_uuid: "33d8bfb0279c474ead72ac856b4ed5a9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 39909 } }
I20260810 07:19:15.153189 8391 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33d8bfb0279c474ead72ac856b4ed5a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e76b7b22184ec2891dffbb4cbecc8b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 36909 } } peers { permanent_uuid: "53402574c2de41a4ae5a87b900d040bf" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 37137 } } peers { permanent_uuid: "33d8bfb0279c474ead72ac856b4ed5a9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 39909 } }
I20260810 07:19:15.153214 8226 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5e76b7b22184ec2891dffbb4cbecc8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5e76b7b22184ec2891dffbb4cbecc8b, State: Initialized, Role: FOLLOWER
I20260810 07:19:15.153278 8226 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5e76b7b22184ec2891dffbb4cbecc8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e76b7b22184ec2891dffbb4cbecc8b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 36909 } } peers { permanent_uuid: "53402574c2de41a4ae5a87b900d040bf" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 37137 } } peers { permanent_uuid: "33d8bfb0279c474ead72ac856b4ed5a9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 39909 } }
I20260810 07:19:15.153477 8391 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33d8bfb0279c474ead72ac856b4ed5a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:15.153584 8226 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5e76b7b22184ec2891dffbb4cbecc8b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:15.153708 8400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33d8bfb0279c474ead72ac856b4ed5a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e76b7b22184ec2891dffbb4cbecc8b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 36909 } } peers { permanent_uuid: "53402574c2de41a4ae5a87b900d040bf" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 37137 } } peers { permanent_uuid: "33d8bfb0279c474ead72ac856b4ed5a9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 39909 } } }
I20260810 07:19:15.153797 8400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33d8bfb0279c474ead72ac856b4ed5a9 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 07:19:15.155591 8424 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b5e76b7b22184ec2891dffbb4cbecc8b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
Aug 10 07:19:15 dist-test-slave-10pc krb5kdc[8138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.125@KRBTEST.COM
W20260810 07:19:15.156950 8424 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:19:15.156555 8152 master_runner.cc:186] Error getting master registration for 127.15.217.125:37137: OK, instance_id { permanent_uuid: "53402574c2de41a4ae5a87b900d040bf" instance_seqno: 1786346354950241 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 07:19:15.159725 8152 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:19:15.159744 8314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:15.150503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35742 (local address 127.15.217.125:37137)
0810 07:19:15.150572 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:15.150574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:15.150583 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:19:15.150609 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:15.150610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:15.150623 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:15.150692 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:15.151354 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.152058 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.153049 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.153349 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.153435 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:15.156546 (+ 3111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:15.156547 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:15.156548 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:15.156557 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:15.158437 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:15.158851 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:15.158853 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:15.158853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:15.158924 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:15.159162 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:15.159163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:15.159163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:15.159313 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:15.159481 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:15.159568 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 07:19:15.159652 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:19:15.155817 8401 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5e76b7b22184ec2891dffbb4cbecc8b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e76b7b22184ec2891dffbb4cbecc8b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 36909 } } peers { permanent_uuid: "53402574c2de41a4ae5a87b900d040bf" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 37137 } } peers { permanent_uuid: "33d8bfb0279c474ead72ac856b4ed5a9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 39909 } } }
I20260810 07:19:15.159791 8401 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5e76b7b22184ec2891dffbb4cbecc8b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:19:15.157465 8317 master_runner.cc:186] Error getting master registration for 127.15.217.125:37137: OK, instance_id { permanent_uuid: "53402574c2de41a4ae5a87b900d040bf" instance_seqno: 1786346354950241 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 07:19:15.160298 8317 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260810 07:19:15.160319 8425 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 33d8bfb0279c474ead72ac856b4ed5a9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:19:15.160545 8425 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:19:15.161398 16229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:19:15.162482 8198 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b5e76b7b22184ec2891dffbb4cbecc8b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37612
I20260810 07:19:15.162597 8198 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5e76b7b22184ec2891dffbb4cbecc8b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 07:19:15.162645 8198 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5e76b7b22184ec2891dffbb4cbecc8b [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:15.163648 8198 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5e76b7b22184ec2891dffbb4cbecc8b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e76b7b22184ec2891dffbb4cbecc8b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 36909 } } peers { permanent_uuid: "53402574c2de41a4ae5a87b900d040bf" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 37137 } } peers { permanent_uuid: "33d8bfb0279c474ead72ac856b4ed5a9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 39909 } }
I20260810 07:19:15.163880 8198 leader_election.cc:288] T 00000000000000000000000000000000 P b5e76b7b22184ec2891dffbb4cbecc8b [CANDIDATE]: Term 1 election: Requested vote from peers 53402574c2de41a4ae5a87b900d040bf (127.15.217.125:37137), 33d8bfb0279c474ead72ac856b4ed5a9 (127.15.217.124:39909)
I20260810 07:19:15.164320 8363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b5e76b7b22184ec2891dffbb4cbecc8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "33d8bfb0279c474ead72ac856b4ed5a9"
I20260810 07:19:15.164435 8363 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33d8bfb0279c474ead72ac856b4ed5a9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:15.165450 8363 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 33d8bfb0279c474ead72ac856b4ed5a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5e76b7b22184ec2891dffbb4cbecc8b in term 1.
I20260810 07:19:15.165766 8170 leader_election.cc:302] T 00000000000000000000000000000000 P b5e76b7b22184ec2891dffbb4cbecc8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d8bfb0279c474ead72ac856b4ed5a9, b5e76b7b22184ec2891dffbb4cbecc8b; no voters:
I20260810 07:19:15.165917 8401 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5e76b7b22184ec2891dffbb4cbecc8b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:15.166127 8401 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5e76b7b22184ec2891dffbb4cbecc8b [term 1 LEADER]: Becoming Leader. State: Replica: b5e76b7b22184ec2891dffbb4cbecc8b, State: Running, Role: LEADER
I20260810 07:19:15.166230 8401 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5e76b7b22184ec2891dffbb4cbecc8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e76b7b22184ec2891dffbb4cbecc8b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 36909 } } peers { permanent_uuid: "53402574c2de41a4ae5a87b900d040bf" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 37137 } } peers { permanent_uuid: "33d8bfb0279c474ead72ac856b4ed5a9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 39909 } }
I20260810 07:19:15.166730 8401 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5e76b7b22184ec2891dffbb4cbecc8b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5e76b7b22184ec2891dffbb4cbecc8b. Latest consensus state: current_term: 1 leader_uuid: "b5e76b7b22184ec2891dffbb4cbecc8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e76b7b22184ec2891dffbb4cbecc8b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 36909 } } peers { permanent_uuid: "53402574c2de41a4ae5a87b900d040bf" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 37137 } } peers { permanent_uuid: "33d8bfb0279c474ead72ac856b4ed5a9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 39909 } } }
I20260810 07:19:15.166857 8401 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5e76b7b22184ec2891dffbb4cbecc8b [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:15.170630 8436 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:15.170904 8436 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:15.179373 8397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:15.168803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41448 (local address 127.15.217.124:39909)
0810 07:19:15.168848 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:15.168851 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:15.170239 (+ 1388us) server_negotiation.cc:407] Connection header received
0810 07:19:15.170416 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:15.170417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:15.170431 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:15.170462 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:15.171812 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.172675 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.173818 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.174210 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.174289 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:15.175920 (+ 1631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:15.175921 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:15.175922 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:15.175930 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:15.177942 (+ 2012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:15.178455 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:15.178457 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:15.178457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:15.178516 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:15.178736 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:15.178737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:15.178737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:15.178882 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:15.178973 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:15.179184 (+ 211us) server_negotiation.cc:299] Negotiation successful
0810 07:19:15.179272 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 07:19:15.185379 8314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:15.169038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35756 (local address 127.15.217.125:37137)
0810 07:19:15.169079 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:15.169081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:15.176612 (+ 7531us) server_negotiation.cc:407] Connection header received
0810 07:19:15.176710 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:15.176711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:15.176726 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:15.176783 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:15.177870 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.178618 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.180117 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.180481 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.180523 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:15.182016 (+ 1493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:15.182017 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:15.182018 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:15.182026 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:15.183863 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:15.184370 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:15.184371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:15.184372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:15.184446 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:15.184698 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:15.184699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:15.184699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:15.184834 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:15.184923 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:15.185176 (+ 253us) server_negotiation.cc:299] Negotiation successful
0810 07:19:15.185270 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 07:19:15.185714 8229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:15.169368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37614 (local address 127.15.217.126:36909)
0810 07:19:15.169484 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:15.169490 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:15.176801 (+ 7311us) server_negotiation.cc:407] Connection header received
0810 07:19:15.176916 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:15.176917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:15.176943 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:15.176977 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:15.177797 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.178526 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.179561 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.179858 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.179908 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:15.181611 (+ 1703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:15.181613 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:15.181613 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:15.181621 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:15.183751 (+ 2130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:15.184433 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:15.184435 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:15.184435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:15.184533 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:15.184838 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:15.184839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:15.184839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:15.185006 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:15.185091 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:15.185442 (+ 351us) server_negotiation.cc:299] Negotiation successful
0810 07:19:15.185616 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 07:19:15.186344 8310 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.217.124" port: 39909 } attempt: 3
Aug 10 07:19:15 dist-test-slave-10pc krb5kdc[8138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346354, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.125@KRBTEST.COM for kudu/127.15.217.124@KRBTEST.COM
I20260810 07:19:15.193617 8363 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 33d8bfb0279c474ead72ac856b4ed5a9 [term 1 FOLLOWER]: Refusing update from remote peer b5e76b7b22184ec2891dffbb4cbecc8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:15.193971 8428 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b5e76b7b22184ec2891dffbb4cbecc8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d8bfb0279c474ead72ac856b4ed5a9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 39909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:15.195094 8312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:15.186610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:39909 (local address 127.0.0.1:41454)
0810 07:19:15.186659 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:15.186670 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:15.186706 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:15.186908 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:15.186909 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:15.187072 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:15.187117 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:15.187119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.188067 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.188068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:15.189070 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:15.189074 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.189635 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.189636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:15.189697 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:15.190179 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:15.190184 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:15.191799 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:15.193835 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:15.193836 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:15.193837 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:15.194060 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:15.194398 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:15.194399 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:15.194399 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:15.194463 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:15.194773 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:15.194774 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:15.194820 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 07:19:15.194885 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 07:19:15.194990 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260810 07:19:15.195343 8400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33d8bfb0279c474ead72ac856b4ed5a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5e76b7b22184ec2891dffbb4cbecc8b. Latest consensus state: current_term: 1 leader_uuid: "b5e76b7b22184ec2891dffbb4cbecc8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e76b7b22184ec2891dffbb4cbecc8b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 36909 } } peers { permanent_uuid: "53402574c2de41a4ae5a87b900d040bf" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 37137 } } peers { permanent_uuid: "33d8bfb0279c474ead72ac856b4ed5a9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 39909 } } }
I20260810 07:19:15.195441 8400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33d8bfb0279c474ead72ac856b4ed5a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:19:15.195736 8397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:15.186700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41454 (local address 127.15.217.124:39909)
0810 07:19:15.186745 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:15.186747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:15.186757 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:19:15.186782 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:15.186784 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:15.186794 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:15.186852 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:15.187202 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.187968 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.189177 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.189497 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.189571 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:15.191918 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:15.191919 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:15.191919 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:15.191927 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:15.193667 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:15.194246 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:15.194247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:15.194247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:15.194307 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:15.194543 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:15.194544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:15.194544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:15.194684 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:15.194744 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:15.195559 (+ 815us) server_negotiation.cc:299] Negotiation successful
0810 07:19:15.195646 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 07:19:15.196857 8428 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5e76b7b22184ec2891dffbb4cbecc8b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b5e76b7b22184ec2891dffbb4cbecc8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e76b7b22184ec2891dffbb4cbecc8b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 36909 } } peers { permanent_uuid: "53402574c2de41a4ae5a87b900d040bf" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 37137 } } peers { permanent_uuid: "33d8bfb0279c474ead72ac856b4ed5a9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 39909 } } }
I20260810 07:19:15.196960 8428 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5e76b7b22184ec2891dffbb4cbecc8b [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:15.197558 8310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53402574c2de41a4ae5a87b900d040bf: Bootstrap starting.
I20260810 07:19:15.198184 8400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33d8bfb0279c474ead72ac856b4ed5a9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b5e76b7b22184ec2891dffbb4cbecc8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e76b7b22184ec2891dffbb4cbecc8b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 36909 } } peers { permanent_uuid: "53402574c2de41a4ae5a87b900d040bf" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 37137 } } peers { permanent_uuid: "33d8bfb0279c474ead72ac856b4ed5a9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 39909 } } }
I20260810 07:19:15.198272 8400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33d8bfb0279c474ead72ac856b4ed5a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:19:15.198323 8310 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53402574c2de41a4ae5a87b900d040bf: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:15.198374 8440 mvcc.cc:205] Tried to move back new op lower bound from 7316874670869884928 to 7316874670761713664. Current Snapshot: MvccSnapshot[applied={T|T < 7316874670869884928}]
I20260810 07:19:15.198642 8310 log.cc:824] T 00000000000000000000000000000000 P 53402574c2de41a4ae5a87b900d040bf: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:15.199414 8441 mvcc.cc:205] Tried to move back new op lower bound from 7316874670869884928 to 7316874670761713664. Current Snapshot: MvccSnapshot[applied={T|T < 7316874670869884928}]
I20260810 07:19:15.199904 8436 catalog_manager.cc:1347] Generated new cluster ID: 5f41bc62aa18497e838c7b7c7c34b5dd
I20260810 07:19:15.199951 8436 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:15.205646 8310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53402574c2de41a4ae5a87b900d040bf: No bootstrap required, opened a new log
I20260810 07:19:15.207945 8310 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53402574c2de41a4ae5a87b900d040bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e76b7b22184ec2891dffbb4cbecc8b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 36909 } } peers { permanent_uuid: "53402574c2de41a4ae5a87b900d040bf" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 37137 } } peers { permanent_uuid: "33d8bfb0279c474ead72ac856b4ed5a9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 39909 } }
I20260810 07:19:15.208098 8310 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53402574c2de41a4ae5a87b900d040bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53402574c2de41a4ae5a87b900d040bf, State: Initialized, Role: FOLLOWER
I20260810 07:19:15.208184 8310 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53402574c2de41a4ae5a87b900d040bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e76b7b22184ec2891dffbb4cbecc8b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 36909 } } peers { permanent_uuid: "53402574c2de41a4ae5a87b900d040bf" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 37137 } } peers { permanent_uuid: "33d8bfb0279c474ead72ac856b4ed5a9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 39909 } }
I20260810 07:19:15.208593 8310 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53402574c2de41a4ae5a87b900d040bf [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:15.210283 8283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b5e76b7b22184ec2891dffbb4cbecc8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "53402574c2de41a4ae5a87b900d040bf"
I20260810 07:19:15.210417 8283 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53402574c2de41a4ae5a87b900d040bf [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:15.211378 8283 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 53402574c2de41a4ae5a87b900d040bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5e76b7b22184ec2891dffbb4cbecc8b in term 1.
I20260810 07:19:15.211802 8281 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 53402574c2de41a4ae5a87b900d040bf [term 1 FOLLOWER]: Refusing update from remote peer b5e76b7b22184ec2891dffbb4cbecc8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:15.212234 8401 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b5e76b7b22184ec2891dffbb4cbecc8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "53402574c2de41a4ae5a87b900d040bf" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 37137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:15.212548 8446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53402574c2de41a4ae5a87b900d040bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e76b7b22184ec2891dffbb4cbecc8b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 36909 } } peers { permanent_uuid: "53402574c2de41a4ae5a87b900d040bf" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 37137 } } peers { permanent_uuid: "33d8bfb0279c474ead72ac856b4ed5a9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 39909 } } }
I20260810 07:19:15.212646 8446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53402574c2de41a4ae5a87b900d040bf [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:19:15.216794 8446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53402574c2de41a4ae5a87b900d040bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5e76b7b22184ec2891dffbb4cbecc8b. Latest consensus state: current_term: 1 leader_uuid: "b5e76b7b22184ec2891dffbb4cbecc8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e76b7b22184ec2891dffbb4cbecc8b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 36909 } } peers { permanent_uuid: "53402574c2de41a4ae5a87b900d040bf" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 37137 } } peers { permanent_uuid: "33d8bfb0279c474ead72ac856b4ed5a9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 39909 } } }
I20260810 07:19:15.216904 8446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53402574c2de41a4ae5a87b900d040bf [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:19:15.216952 8446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53402574c2de41a4ae5a87b900d040bf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b5e76b7b22184ec2891dffbb4cbecc8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e76b7b22184ec2891dffbb4cbecc8b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 36909 } } peers { permanent_uuid: "53402574c2de41a4ae5a87b900d040bf" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 37137 } } peers { permanent_uuid: "33d8bfb0279c474ead72ac856b4ed5a9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 39909 } } }
I20260810 07:19:15.217022 8446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53402574c2de41a4ae5a87b900d040bf [sys.catalog]: This master's current role is: FOLLOWER
W20260810 07:19:15.217552 8458 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 53402574c2de41a4ae5a87b900d040bf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:19:15.217662 8458 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:19:15.218710 8460 mvcc.cc:205] Tried to move back new op lower bound from 7316874670869884928 to 7316874670761713664. Current Snapshot: MvccSnapshot[applied={T|T < 7316874670869884928}]
I20260810 07:19:15.239064 8401 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5e76b7b22184ec2891dffbb4cbecc8b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b5e76b7b22184ec2891dffbb4cbecc8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e76b7b22184ec2891dffbb4cbecc8b" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 36909 } } peers { permanent_uuid: "53402574c2de41a4ae5a87b900d040bf" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 37137 } } peers { permanent_uuid: "33d8bfb0279c474ead72ac856b4ed5a9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 39909 } } }
I20260810 07:19:15.239215 8401 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5e76b7b22184ec2891dffbb4cbecc8b [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:15.242528 8436 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:15.243551 8436 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:15.257256 8436 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b5e76b7b22184ec2891dffbb4cbecc8b: Generated new TSK 0
I20260810 07:19:15.257509 8436 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 07:19:15.287072 8229 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:15.285512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37618 (local address 127.15.217.126:36909)
0810 07:19:15.285559 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:15.285562 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:15.286311 (+ 749us) server_negotiation.cc:407] Connection header received
0810 07:19:15.286431 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:15.286433 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:15.286445 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:15.286477 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:15.286980 (+ 503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37618: BlockingRecv error: recv got EOF from 127.0.0.1:37618 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260810 07:19:15.287887 8314 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:15.285973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35766 (local address 127.15.217.125:37137)
0810 07:19:15.286029 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:15.286031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:15.287342 (+ 1311us) server_negotiation.cc:407] Connection header received
0810 07:19:15.287443 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:15.287444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:15.287457 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:15.287488 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:15.287823 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35766: BlockingRecv error: recv got EOF from 127.0.0.1:35766 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260810 07:19:15.291522 8397 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:15.290725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41464 (local address 127.15.217.124:39909)
0810 07:19:15.290774 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:15.290776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:15.290914 (+ 138us) server_negotiation.cc:407] Connection header received
0810 07:19:15.291029 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:15.291030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:15.291042 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:15.291074 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:15.291456 (+ 382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41464: BlockingRecv error: recv got EOF from 127.0.0.1:41464 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260810 07:19:15.294157 8314 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:15.293263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35770 (local address 127.15.217.125:37137)
0810 07:19:15.293300 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:15.293302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:15.293506 (+ 204us) server_negotiation.cc:407] Connection header received
0810 07:19:15.293621 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:15.293622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:15.293644 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:15.293677 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:15.294106 (+ 429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35770: BlockingRecv error: recv got EOF from 127.0.0.1:35770 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260810 07:19:15.297374 8314 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:15.296495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35772 (local address 127.15.217.125:37137)
0810 07:19:15.296531 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:15.296533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:15.296821 (+ 288us) server_negotiation.cc:407] Connection header received
0810 07:19:15.296920 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:15.296921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:15.296932 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:15.296962 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:15.297330 (+ 368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35772: BlockingRecv error: recv got EOF from 127.0.0.1:35772 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260810 07:19:15.301447 8314 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:15.300631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35776 (local address 127.15.217.125:37137)
0810 07:19:15.300668 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:15.300669 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:15.300908 (+ 239us) server_negotiation.cc:407] Connection header received
0810 07:19:15.301024 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:15.301025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:15.301037 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:15.301068 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:15.301401 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35776: BlockingRecv error: recv got EOF from 127.0.0.1:35776 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260810 07:19:15.307637 8314 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:15.306725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35788 (local address 127.15.217.125:37137)
0810 07:19:15.306771 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:15.306773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:15.307054 (+ 281us) server_negotiation.cc:407] Connection header received
0810 07:19:15.307167 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:15.307168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:15.307181 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:15.307211 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:15.307583 (+ 372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35788: BlockingRecv error: recv got EOF from 127.0.0.1:35788 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260810 07:19:15.318035 8314 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:15.317044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35802 (local address 127.15.217.125:37137)
0810 07:19:15.317084 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:15.317086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:15.317384 (+ 298us) server_negotiation.cc:407] Connection header received
0810 07:19:15.317502 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:15.317504 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:15.317520 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:15.317552 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:15.317950 (+ 398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35802: BlockingRecv error: recv got EOF from 127.0.0.1:35802 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260810 07:19:15.336582 8314 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:15.335532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35810 (local address 127.15.217.125:37137)
0810 07:19:15.335583 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:15.335586 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:15.335895 (+ 309us) server_negotiation.cc:407] Connection header received
0810 07:19:15.336013 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:15.336014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:15.336026 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:15.336056 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:15.336519 (+ 463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35810: BlockingRecv error: recv got EOF from 127.0.0.1:35810 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260810 07:19:15.370887 8314 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:15.370015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35824 (local address 127.15.217.125:37137)
0810 07:19:15.370067 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:15.370070 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:15.370263 (+ 193us) server_negotiation.cc:407] Connection header received
0810 07:19:15.370341 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:15.370343 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:15.370358 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:15.370430 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:15.370809 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35824: BlockingRecv error: recv got EOF from 127.0.0.1:35824 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260810 07:19:15.437197 8314 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:15.436236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35828 (local address 127.15.217.125:37137)
0810 07:19:15.436331 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:15.436334 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:15.436445 (+ 111us) server_negotiation.cc:407] Connection header received
0810 07:19:15.436600 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:15.436601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:15.436613 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:15.436682 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:15.437127 (+ 445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35828: BlockingRecv error: recv got EOF from 127.0.0.1:35828 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260810 07:19:15.567163 8314 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:15.566435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35834 (local address 127.15.217.125:37137)
0810 07:19:15.566542 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:15.566545 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:15.566615 (+ 70us) server_negotiation.cc:407] Connection header received
0810 07:19:15.566668 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:15.566669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:15.566680 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:15.566749 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:15.567085 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35834: BlockingRecv error: recv got EOF from 127.0.0.1:35834 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260810 07:19:15.825178 8314 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:15.824555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35838 (local address 127.15.217.125:37137)
0810 07:19:15.824627 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:15.824629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:15.824644 (+ 15us) server_negotiation.cc:407] Connection header received
0810 07:19:15.824703 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:15.824704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:15.824714 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:15.824779 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:15.825121 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35838: BlockingRecv error: recv got EOF from 127.0.0.1:35838 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 07:19:16.161063 8425 catalog_manager.cc:1259] Loaded cluster ID: 5f41bc62aa18497e838c7b7c7c34b5dd
I20260810 07:19:16.161161 8425 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 33d8bfb0279c474ead72ac856b4ed5a9: loading cluster ID for follower catalog manager: success
I20260810 07:19:16.162021 8425 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 33d8bfb0279c474ead72ac856b4ed5a9: acquiring CA information for follower catalog manager: success
I20260810 07:19:16.162366 8425 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 33d8bfb0279c474ead72ac856b4ed5a9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:19:16.218205 8458 catalog_manager.cc:1259] Loaded cluster ID: 5f41bc62aa18497e838c7b7c7c34b5dd
I20260810 07:19:16.218295 8458 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 53402574c2de41a4ae5a87b900d040bf: loading cluster ID for follower catalog manager: success
I20260810 07:19:16.219152 8458 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 53402574c2de41a4ae5a87b900d040bf: acquiring CA information for follower catalog manager: success
I20260810 07:19:16.219497 8458 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 53402574c2de41a4ae5a87b900d040bf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:19:16.341318 8533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:16.338476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43588 (local address 127.15.217.125:37137)
0810 07:19:16.338617 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:16.338619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:16.338752 (+ 133us) server_negotiation.cc:407] Connection header received
0810 07:19:16.338795 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:16.338796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:16.338856 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:16.338949 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:16.339121 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:16.339661 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:16.340485 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:16.340711 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:16.340804 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:16.340943 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:16.341115 (+ 172us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:16.341180 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:16.341227 (+ 47us) server_negotiation.cc:299] Negotiation successful
0810 07:19:16.341247 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":83,"threads_started":1}
I20260810 07:19:16.346016 8541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:16.343328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43338 (local address 127.15.217.124:39909)
0810 07:19:16.343466 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:16.343469 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:16.343478 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:19:16.343530 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:16.343531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:16.343585 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:16.343683 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:16.343833 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:16.344333 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:16.345149 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:16.345383 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:16.345459 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:16.345651 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:16.345764 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:16.345833 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:16.345874 (+ 41us) server_negotiation.cc:299] Negotiation successful
0810 07:19:16.345893 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":62,"threads_started":1}
I20260810 07:19:16.350798 8533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:16.347947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43596 (local address 127.15.217.125:37137)
0810 07:19:16.347999 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:16.348002 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:16.348073 (+ 71us) server_negotiation.cc:407] Connection header received
0810 07:19:16.348156 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:16.348157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:16.348168 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:16.348246 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:16.348364 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:16.349038 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:16.349931 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:16.350259 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:16.350327 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:16.350525 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:16.350560 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:16.350621 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:16.350680 (+ 59us) server_negotiation.cc:299] Negotiation successful
0810 07:19:16.350706 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 07:19:16.351215 8541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:16.348099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43340 (local address 127.15.217.124:39909)
0810 07:19:16.348302 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:16.348304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:16.348428 (+ 124us) server_negotiation.cc:407] Connection header received
0810 07:19:16.348507 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:16.348508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:16.348518 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:16.348565 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:16.348714 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:16.349449 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:16.350428 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:16.350717 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:16.350773 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:16.350953 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:16.350987 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:16.351015 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:16.351127 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 07:19:16.351148 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179}
I20260810 07:19:16.351702 8549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:16.348012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58208 (local address 127.15.217.126:36909)
0810 07:19:16.348131 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:16.348133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:16.348261 (+ 128us) server_negotiation.cc:407] Connection header received
0810 07:19:16.348357 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:16.348358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:16.348419 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:16.348525 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:16.348771 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:16.349525 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:16.350546 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:16.350859 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:16.350899 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:16.351187 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:16.351304 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:16.351395 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:16.351577 (+ 182us) server_negotiation.cc:299] Negotiation successful
0810 07:19:16.351608 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":62,"threads_started":1}
I20260810 07:19:16.353478 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 8152
I20260810 07:19:16.362358 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 8234
I20260810 07:19:16.370103 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 8317
2026-08-10T07:19:16Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2956 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:16 dist-test-slave-10pc krb5kdc[8553](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:16 dist-test-slave-10pc krb5kdc[8553](info): set up 2 sockets
Aug 10 07:19:16 dist-test-slave-10pc krb5kdc[8553](info): commencing operation
krb5kdc: starting...
W20260810 07:19:17.471630 16229 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/krb5kdc/test-user.keytab.
Aug 10 07:19:17 dist-test-slave-10pc krb5kdc[8553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.217.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
Entry for principal kudu/127.15.217.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab.
I20260810 07:19:17.605428 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:42961
--webserver_interface=127.15.217.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:37443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:42961,127.15.217.125:36253,127.15.217.124:42721
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:17.691478 8566 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 07:19:17.691766 8566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:17.691802 8566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:17.691844 8566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:17.693107 8566 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:17.693182 8566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:17.693212 8566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:17.693240 8566 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:17.693271 8566 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:17.694831 8566 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:37443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:42961,127.15.217.125:36253,127.15.217.124:42721
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.126.keytab
--principal=kudu/127.15.217.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.126:42961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.217.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:17.695123 8566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:17.695367 8566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:17.700407 8573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:17.700417 8575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:17.700403 8572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:17.700716 8566 server_base.cc:1034] running on GCE node
I20260810 07:19:17.701041 8566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:17.701352 8566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:17.702521 8566 hybrid_clock.cc:648] HybridClock initialized: now 1786346357702481 us; error 75 us; skew 500 ppm
Aug 10 07:19:17 dist-test-slave-10pc krb5kdc[8553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346357, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:17.709249 8566 init.cc:377] Logged in from keytab as kudu/127.15.217.126@KRBTEST.COM (short username kudu)
I20260810 07:19:17.709789 8566 webserver.cc:460] Webserver started at http://127.15.217.126:46289/ using document root <none> and password file <none>
I20260810 07:19:17.710026 8566 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:17.710072 8566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:17.710170 8566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:17.711236 8566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-0/data/instance:
uuid: "78994e8b371b423c90a9e64b38d3d591"
format_stamp: "Formatted at 2026-08-10 07:19:17 on dist-test-slave-10pc"
server_key: "1f566dddec763afd8926288af8188445"
server_key_iv: "e5609df6f9b3a81f3c5f7e7d114ceb08"
server_key_version: "encryptionkey@0"
I20260810 07:19:17.711550 8566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-0/wal/instance:
uuid: "78994e8b371b423c90a9e64b38d3d591"
format_stamp: "Formatted at 2026-08-10 07:19:17 on dist-test-slave-10pc"
server_key: "1f566dddec763afd8926288af8188445"
server_key_iv: "e5609df6f9b3a81f3c5f7e7d114ceb08"
server_key_version: "encryptionkey@0"
I20260810 07:19:17.712957 8566 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 07:19:17.713737 8583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:17.713996 8566 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:19:17.714150 8566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-0/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-0/wal
uuid: "78994e8b371b423c90a9e64b38d3d591"
format_stamp: "Formatted at 2026-08-10 07:19:17 on dist-test-slave-10pc"
server_key: "1f566dddec763afd8926288af8188445"
server_key_iv: "e5609df6f9b3a81f3c5f7e7d114ceb08"
server_key_version: "encryptionkey@0"
I20260810 07:19:17.714254 8566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:17.723675 8566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:17.725740 8566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:17.725898 8566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:17.730396 8566 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.126:42961
I20260810 07:19:17.730430 8638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.126:42961 every 8 connection(s)
I20260810 07:19:17.730822 8566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:17.731432 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 8566
I20260810 07:19:17.731566 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-0/wal/instance
I20260810 07:19:17.731806 16229 external_mini_cluster.cc:1428] Setting key 357c47f7c65c10d7a30c02a0d232ae6f
I20260810 07:19:17.732067 8640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:17.734483 8640 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42961 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.15.217.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.125@KRBTEST.COM" created.
Aug 10 07:19:17 dist-test-slave-10pc krb5kdc[8553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346357, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:19:17.748200 8644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:17.735385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32904 (local address 127.15.217.126:42961)
0810 07:19:17.735559 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:17.735566 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:17.735680 (+ 114us) server_negotiation.cc:407] Connection header received
0810 07:19:17.736323 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:17.736325 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:17.736342 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:17.736477 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:17.737061 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:17.737895 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:17.738803 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:17.739132 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:17.739269 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:17.744538 (+ 5269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:17.744548 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:17.744548 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:17.744557 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:17.746383 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:17.746852 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:17.746856 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:17.746856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:17.746953 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:17.747195 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:17.747196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:17.747197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:17.747445 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:17.747539 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:17.747755 (+ 216us) server_negotiation.cc:299] Negotiation successful
0810 07:19:17.747912 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":96,"threads_started":1}
I20260810 07:19:17.748203 8643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:17.735311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42961 (local address 127.0.0.1:32904)
0810 07:19:17.735504 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:17.735555 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:17.736115 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:17.736476 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:17.736478 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:17.736785 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:17.736943 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:17.736948 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:17.738020 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:17.738021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:17.738699 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:17.738701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:17.739236 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:17.739237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:17.739302 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:17.739898 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:17.739907 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:17.744384 (+ 4477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:17.746509 (+ 2125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:17.746518 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:17.746521 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:17.746755 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:17.747028 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:17.747029 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:17.747029 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:17.747126 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:17.747528 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:17.747534 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:17.747602 (+ 68us) client_negotiation.cc:769] Sending connection context
0810 07:19:17.747718 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 07:19:17.747985 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":99,"thread_start_us":97,"threads_started":1}
I20260810 07:19:17.749500 8640 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.217.125" port: 36253 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:19:17.749915 8643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:17.749781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:36253 (local address 127.0.0.1:34620)
0810 07:19:17.749833 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:17.749887 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.125:36253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 07:19:17.750059 8587 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.217.125:36253: connect: Connection refused (error 111)
I20260810 07:19:17.750562 8643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:17.750455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:36253 (local address 127.0.0.1:34636)
0810 07:19:17.750509 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:17.750536 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.125:36253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 07:19:17.750679 8640 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.217.125:36253: Network error: Client connection negotiation failed: client connection to 127.15.217.125:36253: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.125.keytab.
Entry for principal kudu/127.15.217.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.125.keytab.
I20260810 07:19:17.763195 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.125:36253
--webserver_interface=127.15.217.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.125.keytab
--principal=kudu/127.15.217.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:37443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:42961,127.15.217.125:36253,127.15.217.124:42721
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:19:17.799943 8640 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.217.125" port: 36253 } attempt: 1
I20260810 07:19:17.800594 8643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:17.800403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:36253 (local address 127.0.0.1:34650)
0810 07:19:17.800514 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:17.800560 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.125:36253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260810 07:19:17.801028 8643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:17.800930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:36253 (local address 127.0.0.1:34654)
0810 07:19:17.800983 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:17.801010 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.125:36253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260810 07:19:17.801158 8640 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.217.125:36253: Network error: Client connection negotiation failed: client connection to 127.15.217.125:36253: connect: Connection refused (error 111)
W20260810 07:19:17.851210 8647 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 07:19:17.851432 8647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:17.851450 8647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:17.851470 8647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:17.852689 8647 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:17.852730 8647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:17.852744 8647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:17.852757 8647 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:17.852810 8647 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:17.854379 8647 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:37443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:42961,127.15.217.125:36253,127.15.217.124:42721
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.125.keytab
--principal=kudu/127.15.217.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.125:36253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.217.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:17.854655 8647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:17.854910 8647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:17.859795 8652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:17.859822 8655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:17.859799 8653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:17.860096 8647 server_base.cc:1034] running on GCE node
I20260810 07:19:17.860311 8647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:17.860617 8647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:17.861766 8647 hybrid_clock.cc:648] HybridClock initialized: now 1786346357861756 us; error 32 us; skew 500 ppm
Aug 10 07:19:17 dist-test-slave-10pc krb5kdc[8553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346357, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:17.864250 8647 init.cc:377] Logged in from keytab as kudu/127.15.217.125@KRBTEST.COM (short username kudu)
I20260810 07:19:17.864763 8647 webserver.cc:460] Webserver started at http://127.15.217.125:44741/ using document root <none> and password file <none>
I20260810 07:19:17.865008 8647 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:17.865085 8647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:17.865236 8647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:17.866312 8647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-1/data/instance:
uuid: "26a724e260724a48988c84c7ccfd6e98"
format_stamp: "Formatted at 2026-08-10 07:19:17 on dist-test-slave-10pc"
server_key: "ad8633c583cf7795d40d5cf03c1c3a35"
server_key_iv: "8620c5b91ec274f9ca9f1444a7b3801b"
server_key_version: "encryptionkey@0"
I20260810 07:19:17.866720 8647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-1/wal/instance:
uuid: "26a724e260724a48988c84c7ccfd6e98"
format_stamp: "Formatted at 2026-08-10 07:19:17 on dist-test-slave-10pc"
server_key: "ad8633c583cf7795d40d5cf03c1c3a35"
server_key_iv: "8620c5b91ec274f9ca9f1444a7b3801b"
server_key_version: "encryptionkey@0"
I20260810 07:19:17.868099 8647 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 07:19:17.868909 8662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:17.869099 8647 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:17.869210 8647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-1/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-1/wal
uuid: "26a724e260724a48988c84c7ccfd6e98"
format_stamp: "Formatted at 2026-08-10 07:19:17 on dist-test-slave-10pc"
server_key: "ad8633c583cf7795d40d5cf03c1c3a35"
server_key_iv: "8620c5b91ec274f9ca9f1444a7b3801b"
server_key_version: "encryptionkey@0"
I20260810 07:19:17.869287 8647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:17.869350 8640 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.217.125" port: 36253 } attempt: 2
I20260810 07:19:17.869773 8643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:17.869658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:36253 (local address 127.0.0.1:34670)
0810 07:19:17.869715 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:17.869749 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.125:36253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 07:19:17.870100 8643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:17.870003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:36253 (local address 127.0.0.1:34674)
0810 07:19:17.870047 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:17.870072 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.125:36253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260810 07:19:17.870196 8640 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.217.125:36253: Network error: Client connection negotiation failed: client connection to 127.15.217.125:36253: connect: Connection refused (error 111)
I20260810 07:19:17.881229 8647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:17.883163 8647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:17.883339 8647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:17.887861 8647 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.125:36253
I20260810 07:19:17.887920 8718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.125:36253 every 8 connection(s)
I20260810 07:19:17.888324 8647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-1/data/info.pb
I20260810 07:19:17.889324 8719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:17.889587 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 8647
I20260810 07:19:17.889708 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-1/wal/instance
I20260810 07:19:17.889905 16229 external_mini_cluster.cc:1428] Setting key 87ac19efa9e55dbffe2776da1636101f
I20260810 07:19:17.891053 8719 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42961 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:19:17 dist-test-slave-10pc krb5kdc[8553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346357, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.125@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.217.124@KRBTEST.COM; defaulting to no policy
I20260810 07:19:17.901028 8644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:17.891847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32908 (local address 127.15.217.126:42961)
0810 07:19:17.891908 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:17.891910 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:17.892121 (+ 211us) server_negotiation.cc:407] Connection header received
0810 07:19:17.892651 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:17.892652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:17.892670 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:17.892752 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:17.893446 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:17.894209 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:17.895176 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:17.895542 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:17.895630 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:17.897935 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:17.897937 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:17.897937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:17.897946 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:17.899581 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:17.899974 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:17.899975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:17.899975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:17.900039 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:17.900265 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:17.900266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:17.900266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:17.900491 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:17.900564 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:17.900881 (+ 317us) server_negotiation.cc:299] Negotiation successful
0810 07:19:17.900961 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 07:19:17.901117 8722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:17.891816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42961 (local address 127.0.0.1:32908)
0810 07:19:17.891980 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:17.892023 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:17.892553 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:17.892805 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:17.892808 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:17.893138 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:17.893350 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:17.893354 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:17.894314 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:17.894315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:17.895085 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:17.895087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:17.895642 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:17.895643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:17.895757 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:17.896352 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:17.896361 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:17.897837 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:17.899670 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:17.899678 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:17.899680 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:17.899903 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:17.900103 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:17.900104 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:17.900104 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:17.900208 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:17.900564 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:17.900571 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:17.900728 (+ 157us) client_negotiation.cc:769] Sending connection context
0810 07:19:17.900822 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 07:19:17.900950 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":82,"threads_started":1}
I20260810 07:19:17.901819 8719 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.217.125" port: 36253 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.217.124@KRBTEST.COM" created.
Aug 10 07:19:17 dist-test-slave-10pc krb5kdc[8553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346357, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.125@KRBTEST.COM for kudu/127.15.217.125@KRBTEST.COM
I20260810 07:19:17.910080 8722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:17.902115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:36253 (local address 127.0.0.1:34684)
0810 07:19:17.902151 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:17.902162 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:17.902202 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:17.902494 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:17.902495 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:17.902637 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:17.902691 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:17.902693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:17.903549 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:17.903550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:17.904280 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:17.904283 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:17.904920 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:17.904921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:17.905024 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:17.905462 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:17.905467 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:17.906644 (+ 1177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:17.908940 (+ 2296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:17.908941 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:17.908942 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:17.909105 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:17.909342 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:17.909343 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:17.909343 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:17.909398 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:17.909768 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:17.909769 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:17.909810 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 07:19:17.909868 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 07:19:17.909954 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260810 07:19:17.910187 8723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:17.902198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34684 (local address 127.15.217.125:36253)
0810 07:19:17.902352 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:17.902357 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:17.902370 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:19:17.902387 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:17.902389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:17.902399 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:17.902493 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:17.902778 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:17.903450 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:17.904389 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:17.904832 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:17.904904 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:17.906747 (+ 1843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:17.906759 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:17.906760 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:17.906769 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:17.908847 (+ 2078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:17.909193 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:17.909198 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:17.909198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:17.909269 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:17.909465 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:17.909466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:17.909467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:17.909687 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:17.909788 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:17.909909 (+ 121us) server_negotiation.cc:299] Negotiation successful
0810 07:19:17.910025 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":116,"thread_start_us":89,"threads_started":1}
I20260810 07:19:17.911108 8719 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42721 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:19:17.911486 8722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:17.911358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:42721 (local address 127.0.0.1:52874)
0810 07:19:17.911413 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:17.911463 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.124:42721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260810 07:19:17.911612 8667 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.217.124:42721: connect: Connection refused (error 111)
I20260810 07:19:17.912097 8722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:17.911991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:42721 (local address 127.0.0.1:52884)
0810 07:19:17.912046 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:17.912077 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.124:42721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260810 07:19:17.912251 8719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.217.124:42721: Network error: Client connection negotiation failed: client connection to 127.15.217.124:42721: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.217.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.124.keytab.
Entry for principal kudu/127.15.217.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.124.keytab.
I20260810 07:19:17.921439 16229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvoWU8l/build/release/bin/kudu
/tmp/dist-test-taskvoWU8l/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.124:42721
--webserver_interface=127.15.217.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.124.keytab
--principal=kudu/127.15.217.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.217.84:37443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.217.126:42961,127.15.217.125:36253,127.15.217.124:42721
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:19:17.961360 8640 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.217.125" port: 36253 } attempt: 3
I20260810 07:19:17.961511 8719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42721 } attempt: 1
I20260810 07:19:17.962064 8722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:17.961902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:42721 (local address 127.0.0.1:52894)
0810 07:19:17.961974 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:17.962019 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.124:42721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260810 07:19:17.962472 8722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:17.962382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:42721 (local address 127.0.0.1:52910)
0810 07:19:17.962420 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:17.962449 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.124:42721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260810 07:19:17.962596 8719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.217.124:42721: Network error: Client connection negotiation failed: client connection to 127.15.217.124:42721: connect: Connection refused (error 111)
Aug 10 07:19:17 dist-test-slave-10pc krb5kdc[8553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346357, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for kudu/127.15.217.125@KRBTEST.COM
I20260810 07:19:17.969013 8723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:17.961949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34686 (local address 127.15.217.125:36253)
0810 07:19:17.961989 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:17.961992 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:17.962001 (+ 9us) server_negotiation.cc:407] Connection header received
0810 07:19:17.962027 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:17.962028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:17.962039 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:17.962119 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:17.962618 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:17.963149 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:17.963891 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:17.964131 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:17.964213 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:17.966390 (+ 2177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:17.966391 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:17.966391 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:17.966399 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:17.967807 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:17.968171 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:17.968172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:17.968172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:17.968233 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:17.968441 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:17.968442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:17.968442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:17.968595 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:17.968674 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:17.968837 (+ 163us) server_negotiation.cc:299] Negotiation successful
0810 07:19:17.968918 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 07:19:17.969017 8643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:17.961795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:36253 (local address 127.0.0.1:34686)
0810 07:19:17.961874 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:17.961889 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:17.961948 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:17.962130 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:17.962131 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:17.962389 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:17.962474 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:17.962477 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:17.963258 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:17.963259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:17.963793 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:17.963795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:17.964211 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:17.964212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:17.964300 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:17.964818 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:17.964823 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:17.966297 (+ 1474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:17.967896 (+ 1599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:17.967897 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:17.967897 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:17.968085 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:17.968306 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:17.968307 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:17.968307 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:17.968370 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:17.968665 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:17.968666 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:17.968716 (+ 50us) client_negotiation.cc:769] Sending connection context
0810 07:19:17.968810 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 07:19:17.968918 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 07:19:17.969643 8640 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42721 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:19:17.970007 8643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:17.969912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:42721 (local address 127.0.0.1:52922)
0810 07:19:17.969960 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:17.969989 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.124:42721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260810 07:19:17.970333 8643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:17.970253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:42721 (local address 127.0.0.1:52936)
0810 07:19:17.970294 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:17.970318 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.124:42721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260810 07:19:17.970423 8640 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.217.124:42721: Network error: Client connection negotiation failed: client connection to 127.15.217.124:42721: connect: Connection refused (error 111)
I20260810 07:19:18.001583 8640 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42721 } attempt: 1
I20260810 07:19:18.002130 8643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:18.001984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:42721 (local address 127.0.0.1:52948)
0810 07:19:18.002055 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:18.002098 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.124:42721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 07:19:18.002589 8643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:18.002460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:42721 (local address 127.0.0.1:52960)
0810 07:19:18.002530 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:18.002565 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.124:42721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 07:19:18.002712 8640 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.217.124:42721: Network error: Client connection negotiation failed: client connection to 127.15.217.124:42721: connect: Connection refused (error 111)
W20260810 07:19:18.011618 8726 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 07:19:18.011852 8726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:18.011870 8726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:18.011893 8726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:18.013219 8726 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:18.013269 8726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:18.013283 8726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:18.013298 8726 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:18.013309 8726 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:18.014869 8726 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.217.84:37443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.217.126:42961,127.15.217.125:36253,127.15.217.124:42721
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/krb5kdc/kudu_127.15.217.124.keytab
--principal=kudu/127.15.217.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.217.124:42721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.217.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 06:23:07 UTC on ccycloud.quasar-pwrldj.root.comops.site
build id 4577110
I20260810 07:19:18.015172 8726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:18.015424 8726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:18.020929 8732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:18.020949 8733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:18.021165 8726 server_base.cc:1034] running on GCE node
W20260810 07:19:18.020929 8735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:18.021503 8726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:18.021850 8726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:18.023033 8726 hybrid_clock.cc:648] HybridClock initialized: now 1786346358023016 us; error 45 us; skew 500 ppm
Aug 10 07:19:18 dist-test-slave-10pc krb5kdc[8553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346358, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:18.025821 8726 init.cc:377] Logged in from keytab as kudu/127.15.217.124@KRBTEST.COM (short username kudu)
I20260810 07:19:18.026337 8726 webserver.cc:460] Webserver started at http://127.15.217.124:37351/ using document root <none> and password file <none>
I20260810 07:19:18.026597 8726 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:18.026669 8726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:18.026834 8726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:18.028030 8726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-2/data/instance:
uuid: "e483b018db7b400ea03a6bcabfa946c9"
format_stamp: "Formatted at 2026-08-10 07:19:18 on dist-test-slave-10pc"
server_key: "5286e21a50b6f1e2a9d8a8f73bad44a6"
server_key_iv: "0f5e96fbf90cecbdbae671db534f9d5c"
server_key_version: "encryptionkey@0"
I20260810 07:19:18.028502 8726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-2/wal/instance:
uuid: "e483b018db7b400ea03a6bcabfa946c9"
format_stamp: "Formatted at 2026-08-10 07:19:18 on dist-test-slave-10pc"
server_key: "5286e21a50b6f1e2a9d8a8f73bad44a6"
server_key_iv: "0f5e96fbf90cecbdbae671db534f9d5c"
server_key_version: "encryptionkey@0"
I20260810 07:19:18.030098 8726 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:19:18.030761 8719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42721 } attempt: 2
I20260810 07:19:18.030978 8743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:18.031289 8722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:18.031139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:42721 (local address 127.0.0.1:52966)
0810 07:19:18.031219 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:18.031258 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.124:42721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260810 07:19:18.031454 8726 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:19:18.031559 8726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-2/data,/tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-2/wal
uuid: "e483b018db7b400ea03a6bcabfa946c9"
format_stamp: "Formatted at 2026-08-10 07:19:18 on dist-test-slave-10pc"
server_key: "5286e21a50b6f1e2a9d8a8f73bad44a6"
server_key_iv: "0f5e96fbf90cecbdbae671db534f9d5c"
server_key_version: "encryptionkey@0"
I20260810 07:19:18.031670 8726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:18.031735 8722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:18.031620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:42721 (local address 127.0.0.1:52968)
0810 07:19:18.031674 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:18.031710 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.217.124:42721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 07:19:18.031824 8719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.217.124:42721: Network error: Client connection negotiation failed: client connection to 127.15.217.124:42721: connect: Connection refused (error 111)
I20260810 07:19:18.041867 8726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:18.043972 8726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:18.044116 8726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:18.048579 8726 rpc_server.cc:307] RPC server started. Bound to: 127.15.217.124:42721
I20260810 07:19:18.048642 8797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.217.124:42721 every 8 connection(s)
I20260810 07:19:18.049017 8726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-2/data/info.pb
I20260810 07:19:18.050007 8798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:18.051677 8798 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42961 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:19:18.057814 16229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvoWU8l/build/release/bin/kudu as pid 8726
I20260810 07:19:18.057964 16229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvoWU8l/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346081759552-16229-0/minicluster-data/master-2/wal/instance
I20260810 07:19:18.058199 16229 external_mini_cluster.cc:1428] Setting key 78acc8307a9cdbc883f282dd11876e8c
Aug 10 07:19:18 dist-test-slave-10pc krb5kdc[8553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346358, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.124@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:19:18.062208 8644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:18.052630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32914 (local address 127.15.217.126:42961)
0810 07:19:18.052719 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:18.052721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:18.053115 (+ 394us) server_negotiation.cc:407] Connection header received
0810 07:19:18.053840 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:18.053842 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:18.053856 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:18.053916 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:18.054639 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:18.055136 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:18.056219 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:18.056433 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:18.056572 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:18.059088 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:18.059089 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:18.059089 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:18.059097 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:18.060688 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:18.061122 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:18.061124 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:18.061124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:18.061189 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:18.061457 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:18.061459 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:18.061459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:18.061585 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:18.061659 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:18.061982 (+ 323us) server_negotiation.cc:299] Negotiation successful
0810 07:19:18.062087 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 07:19:18.062373 8800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:18.052565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.126:42961 (local address 127.0.0.1:32914)
0810 07:19:18.052902 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:18.052976 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:18.053707 (+ 731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:18.053996 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:18.054001 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:18.054304 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:18.054529 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:18.054534 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:18.055247 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:18.055248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:18.056113 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:18.056116 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:18.056599 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:18.056600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:18.056737 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:18.057243 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:18.057263 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:18.058937 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:18.060786 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:18.060795 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:18.060798 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:18.061042 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:18.061266 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:18.061267 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:18.061267 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:18.061388 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:18.061663 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:18.061671 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:18.061819 (+ 148us) client_negotiation.cc:769] Sending connection context
0810 07:19:18.061941 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 07:19:18.062118 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":218,"threads_started":1}
I20260810 07:19:18.063283 8798 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.217.125" port: 36253 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:19:18 dist-test-slave-10pc krb5kdc[8553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.124@KRBTEST.COM
I20260810 07:19:18.067435 8803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:18.058640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52980 (local address 127.15.217.124:42721)
0810 07:19:18.058845 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:18.058850 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:18.058901 (+ 51us) server_negotiation.cc:407] Connection header received
0810 07:19:18.058979 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:18.058982 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:18.058997 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:18.059081 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:18.059711 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:18.060436 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:18.061245 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:18.061569 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:18.061660 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:18.064410 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:18.064418 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:18.064418 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:18.064425 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:18.066110 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:18.066478 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:18.066481 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:18.066481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:18.066544 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:18.066791 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:18.066792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:18.066792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:18.066986 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:18.067188 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:18.067233 (+ 45us) server_negotiation.cc:299] Negotiation successful
0810 07:19:18.067315 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":97,"threads_started":1}
Aug 10 07:19:18 dist-test-slave-10pc krb5kdc[8553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346358, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.124@KRBTEST.COM for kudu/127.15.217.125@KRBTEST.COM
I20260810 07:19:18.071897 8800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:18.063620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.125:36253 (local address 127.0.0.1:34690)
0810 07:19:18.063682 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:18.063694 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:18.063739 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:18.063932 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:18.063932 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:18.064090 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:18.064151 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:18.064152 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:18.065140 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:18.065141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:18.065776 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:18.065779 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:18.066297 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:18.066298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:18.066377 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:18.066787 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:18.066792 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:18.068112 (+ 1320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:18.070648 (+ 2536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:18.070648 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:18.070649 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:18.070820 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:18.071060 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:18.071061 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:18.071061 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:18.071129 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:18.071509 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:18.071510 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:18.071574 (+ 64us) client_negotiation.cc:769] Sending connection context
0810 07:19:18.071669 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 07:19:18.071778 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 07:19:18.071907 8723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:18.063739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34690 (local address 127.15.217.125:36253)
0810 07:19:18.063809 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:18.063811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:18.063819 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:19:18.063844 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:18.063845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:18.063854 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:18.063927 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:18.064271 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:18.065053 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:18.065867 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:18.066223 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:18.066292 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:18.068203 (+ 1911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:18.068204 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:18.068205 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:18.068211 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:18.070555 (+ 2344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:18.070928 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:18.070929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:18.070929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:18.070997 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:18.071225 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:18.071227 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:18.071227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:18.071427 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:18.071520 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:18.071682 (+ 162us) server_negotiation.cc:299] Negotiation successful
0810 07:19:18.071789 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 07:19:18.072391 8798 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42721 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:19:18 dist-test-slave-10pc krb5kdc[8553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.126@KRBTEST.COM
I20260810 07:19:18.077255 8644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:18.068310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32920 (local address 127.15.217.126:42961)
0810 07:19:18.068361 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:18.068363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:18.068428 (+ 65us) server_negotiation.cc:407] Connection header received
0810 07:19:18.068538 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:18.068539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:18.068549 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:18.068602 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:18.069246 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:18.069907 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:18.070922 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:18.071224 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:18.071302 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:18.074262 (+ 2960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:18.074264 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:18.074264 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:18.074271 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:18.075931 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:18.076321 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:18.076322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:18.076323 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:18.076404 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:18.076684 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:18.076685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:18.076686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:18.076820 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:18.077043 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:18.077078 (+ 35us) server_negotiation.cc:299] Negotiation successful
0810 07:19:18.077143 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
Aug 10 07:19:18 dist-test-slave-10pc krb5kdc[8553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346358, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.124@KRBTEST.COM for kudu/127.15.217.124@KRBTEST.COM
I20260810 07:19:18.077870 8640 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42721 } attempt: 2
I20260810 07:19:18.080914 8800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:18.072727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:42721 (local address 127.0.0.1:52982)
0810 07:19:18.072760 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:18.072770 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:18.072808 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:18.073667 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:18.073668 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:18.073789 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:18.073822 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:18.073823 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:18.074762 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:18.074763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:18.075544 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:18.075547 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:18.075925 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:18.075926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:18.075992 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:18.076437 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:18.076442 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:18.077923 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:18.079865 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:18.079866 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:18.079866 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:18.080087 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:18.080281 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:18.080281 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:18.080281 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:18.080335 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:18.080634 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:18.080634 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:18.080676 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 07:19:18.080735 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 07:19:18.080820 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260810 07:19:18.081236 8803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:18.072764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52982 (local address 127.15.217.124:42721)
0810 07:19:18.072831 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:18.072834 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:18.072842 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:19:18.072865 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:18.072866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:18.072878 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:18.072943 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:18.073893 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:18.074401 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:18.075650 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:18.075847 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:18.075902 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:18.078019 (+ 2117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:18.078020 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:18.078021 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:18.078027 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:18.079787 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:18.080148 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:18.080148 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:18.080149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:18.080201 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:18.080403 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:18.080404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:18.080405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:18.080552 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:18.080613 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:18.081088 (+ 475us) server_negotiation.cc:299] Negotiation successful
0810 07:19:18.081155 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
Aug 10 07:19:18 dist-test-slave-10pc krb5kdc[8553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346357, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.126@KRBTEST.COM for kudu/127.15.217.124@KRBTEST.COM
I20260810 07:19:18.083385 8798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e483b018db7b400ea03a6bcabfa946c9: Bootstrap starting.
Aug 10 07:19:18 dist-test-slave-10pc krb5kdc[8553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.217.125@KRBTEST.COM
I20260810 07:19:18.084204 8798 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e483b018db7b400ea03a6bcabfa946c9: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:18.084604 8798 log.cc:824] T 00000000000000000000000000000000 P e483b018db7b400ea03a6bcabfa946c9: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:18.085657 8798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e483b018db7b400ea03a6bcabfa946c9: No bootstrap required, opened a new log
I20260810 07:19:18.086970 8643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:18.078119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:42721 (local address 127.0.0.1:52998)
0810 07:19:18.078164 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:18.078175 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:18.078214 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:18.078528 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:18.078529 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:18.078678 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:18.078710 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:18.078712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:18.079411 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:18.079412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:18.079997 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:18.080000 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:18.080987 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:18.080989 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:18.081048 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:18.081546 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:18.081550 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:18.083014 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:18.085075 (+ 2061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:18.085076 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:18.085077 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:18.085313 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:18.085556 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:18.085557 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:18.085557 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:18.085630 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:18.086634 (+ 1004us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:18.086636 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:18.086682 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 07:19:18.086751 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 07:19:18.086868 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260810 07:19:18.087390 8804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:18.078209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52998 (local address 127.15.217.124:42721)
0810 07:19:18.078394 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:18.078395 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:18.078402 (+ 7us) server_negotiation.cc:407] Connection header received
0810 07:19:18.078428 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:18.078428 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:18.078436 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:18.078508 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:18.078795 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:18.079328 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:18.080086 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:18.080882 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:18.080966 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:18.083117 (+ 2151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:18.083119 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:18.083119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:18.083127 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:18.084960 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:18.085409 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:18.085410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:18.085410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:18.085472 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:18.085815 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:18.085816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:18.085816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:18.086542 (+ 726us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:18.086628 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:18.087202 (+ 574us) server_negotiation.cc:299] Negotiation successful
0810 07:19:18.087287 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":116,"threads_started":1}
I20260810 07:19:18.087764 8798 raft_consensus.cc:348] T 00000000000000000000000000000000 P e483b018db7b400ea03a6bcabfa946c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78994e8b371b423c90a9e64b38d3d591" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42961 } } peers { permanent_uuid: "26a724e260724a48988c84c7ccfd6e98" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 36253 } } peers { permanent_uuid: "e483b018db7b400ea03a6bcabfa946c9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42721 } }
I20260810 07:19:18.087946 8798 raft_consensus.cc:729] T 00000000000000000000000000000000 P e483b018db7b400ea03a6bcabfa946c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e483b018db7b400ea03a6bcabfa946c9, State: Initialized, Role: FOLLOWER
I20260810 07:19:18.088024 8723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:18.078582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34702 (local address 127.15.217.125:36253)
0810 07:19:18.078670 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:18.078672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:18.078746 (+ 74us) server_negotiation.cc:407] Connection header received
0810 07:19:18.078833 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:18.078835 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:18.078847 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:18.078880 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:18.079755 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:18.080426 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:18.081821 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:18.082057 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:18.082134 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:18.084807 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:18.084809 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:18.084809 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:18.084818 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:18.086351 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:18.086752 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:18.086753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:18.086753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:18.086816 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:18.087062 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:18.087063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:18.087063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:18.087203 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:18.087785 (+ 582us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:18.087830 (+ 45us) server_negotiation.cc:299] Negotiation successful
0810 07:19:18.087907 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 07:19:18.088038 8798 consensus_queue.cc:260] T 00000000000000000000000000000000 P e483b018db7b400ea03a6bcabfa946c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78994e8b371b423c90a9e64b38d3d591" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42961 } } peers { permanent_uuid: "26a724e260724a48988c84c7ccfd6e98" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 36253 } } peers { permanent_uuid: "e483b018db7b400ea03a6bcabfa946c9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42721 } }
I20260810 07:19:18.088384 8798 sys_catalog.cc:564] T 00000000000000000000000000000000 P e483b018db7b400ea03a6bcabfa946c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:18.088995 8640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78994e8b371b423c90a9e64b38d3d591: Bootstrap starting.
I20260810 07:19:18.089612 16229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:19:18.089736 8640 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78994e8b371b423c90a9e64b38d3d591: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:18.090029 8640 log.cc:824] T 00000000000000000000000000000000 P 78994e8b371b423c90a9e64b38d3d591: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:18.090111 8806 sys_catalog.cc:455] T 00000000000000000000000000000000 P e483b018db7b400ea03a6bcabfa946c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78994e8b371b423c90a9e64b38d3d591" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42961 } } peers { permanent_uuid: "26a724e260724a48988c84c7ccfd6e98" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 36253 } } peers { permanent_uuid: "e483b018db7b400ea03a6bcabfa946c9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42721 } } }
I20260810 07:19:18.090214 8806 sys_catalog.cc:458] T 00000000000000000000000000000000 P e483b018db7b400ea03a6bcabfa946c9 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 07:19:18.090891 8817 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e483b018db7b400ea03a6bcabfa946c9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:19:18.091002 8817 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:19:18.091012 8640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78994e8b371b423c90a9e64b38d3d591: No bootstrap required, opened a new log
I20260810 07:19:18.090727 8610 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "78994e8b371b423c90a9e64b38d3d591"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32920
I20260810 07:19:18.092849 8640 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78994e8b371b423c90a9e64b38d3d591 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78994e8b371b423c90a9e64b38d3d591" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42961 } } peers { permanent_uuid: "26a724e260724a48988c84c7ccfd6e98" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 36253 } } peers { permanent_uuid: "e483b018db7b400ea03a6bcabfa946c9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42721 } }
I20260810 07:19:18.093001 8640 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78994e8b371b423c90a9e64b38d3d591 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78994e8b371b423c90a9e64b38d3d591, State: Initialized, Role: FOLLOWER
I20260810 07:19:18.093075 8640 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78994e8b371b423c90a9e64b38d3d591 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78994e8b371b423c90a9e64b38d3d591" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42961 } } peers { permanent_uuid: "26a724e260724a48988c84c7ccfd6e98" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 36253 } } peers { permanent_uuid: "e483b018db7b400ea03a6bcabfa946c9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42721 } }
I20260810 07:19:18.093410 8640 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78994e8b371b423c90a9e64b38d3d591 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:18.093444 8610 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78994e8b371b423c90a9e64b38d3d591 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 07:19:18.093487 8610 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78994e8b371b423c90a9e64b38d3d591 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:18.094501 8610 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78994e8b371b423c90a9e64b38d3d591 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78994e8b371b423c90a9e64b38d3d591" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42961 } } peers { permanent_uuid: "26a724e260724a48988c84c7ccfd6e98" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 36253 } } peers { permanent_uuid: "e483b018db7b400ea03a6bcabfa946c9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42721 } }
I20260810 07:19:18.091840 8726 master_runner.cc:186] Error getting master registration for 127.15.217.126:42961: OK, instance_id { permanent_uuid: "78994e8b371b423c90a9e64b38d3d591" instance_seqno: 1786346357729149 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 07:19:18.094919 8726 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:19:18.094805 8610 leader_election.cc:288] T 00000000000000000000000000000000 P 78994e8b371b423c90a9e64b38d3d591 [CANDIDATE]: Term 1 election: Requested vote from peers 26a724e260724a48988c84c7ccfd6e98 (127.15.217.125:36253), e483b018db7b400ea03a6bcabfa946c9 (127.15.217.124:42721)
I20260810 07:19:18.095481 8771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "78994e8b371b423c90a9e64b38d3d591" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e483b018db7b400ea03a6bcabfa946c9"
I20260810 07:19:18.095604 8771 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e483b018db7b400ea03a6bcabfa946c9 [term 0 FOLLOWER]: Advancing to term 1
W20260810 07:19:18.095968 8831 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 78994e8b371b423c90a9e64b38d3d591: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:19:18.096029 8831 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:19:18.093386 8820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78994e8b371b423c90a9e64b38d3d591 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78994e8b371b423c90a9e64b38d3d591" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42961 } } peers { permanent_uuid: "26a724e260724a48988c84c7ccfd6e98" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 36253 } } peers { permanent_uuid: "e483b018db7b400ea03a6bcabfa946c9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42721 } } }
I20260810 07:19:18.096771 8771 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e483b018db7b400ea03a6bcabfa946c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78994e8b371b423c90a9e64b38d3d591 in term 1.
I20260810 07:19:18.096796 8820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78994e8b371b423c90a9e64b38d3d591 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:19:18.097044 8587 leader_election.cc:302] T 00000000000000000000000000000000 P 78994e8b371b423c90a9e64b38d3d591 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78994e8b371b423c90a9e64b38d3d591, e483b018db7b400ea03a6bcabfa946c9; no voters:
I20260810 07:19:18.097173 8820 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78994e8b371b423c90a9e64b38d3d591 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:18.097304 8820 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78994e8b371b423c90a9e64b38d3d591 [term 1 LEADER]: Becoming Leader. State: Replica: 78994e8b371b423c90a9e64b38d3d591, State: Running, Role: LEADER
I20260810 07:19:18.097388 8820 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78994e8b371b423c90a9e64b38d3d591 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78994e8b371b423c90a9e64b38d3d591" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42961 } } peers { permanent_uuid: "26a724e260724a48988c84c7ccfd6e98" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 36253 } } peers { permanent_uuid: "e483b018db7b400ea03a6bcabfa946c9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42721 } }
I20260810 07:19:18.098676 8566 master_runner.cc:186] Error getting master registration for 127.15.217.125:36253: OK, instance_id { permanent_uuid: "26a724e260724a48988c84c7ccfd6e98" instance_seqno: 1786346357886567 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 07:19:18.101397 8566 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:19:18.101507 8820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78994e8b371b423c90a9e64b38d3d591 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78994e8b371b423c90a9e64b38d3d591. Latest consensus state: current_term: 1 leader_uuid: "78994e8b371b423c90a9e64b38d3d591" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78994e8b371b423c90a9e64b38d3d591" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42961 } } peers { permanent_uuid: "26a724e260724a48988c84c7ccfd6e98" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 36253 } } peers { permanent_uuid: "e483b018db7b400ea03a6bcabfa946c9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42721 } } }
I20260810 07:19:18.101600 8820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78994e8b371b423c90a9e64b38d3d591 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:18.101907 8841 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:18.102216 8841 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:18.104257 8771 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e483b018db7b400ea03a6bcabfa946c9 [term 1 FOLLOWER]: Refusing update from remote peer 78994e8b371b423c90a9e64b38d3d591: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:18.104761 8820 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 78994e8b371b423c90a9e64b38d3d591 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e483b018db7b400ea03a6bcabfa946c9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:18.106472 8806 sys_catalog.cc:455] T 00000000000000000000000000000000 P e483b018db7b400ea03a6bcabfa946c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78994e8b371b423c90a9e64b38d3d591. Latest consensus state: current_term: 1 leader_uuid: "78994e8b371b423c90a9e64b38d3d591" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78994e8b371b423c90a9e64b38d3d591" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42961 } } peers { permanent_uuid: "26a724e260724a48988c84c7ccfd6e98" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 36253 } } peers { permanent_uuid: "e483b018db7b400ea03a6bcabfa946c9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42721 } } }
I20260810 07:19:18.106583 8806 sys_catalog.cc:458] T 00000000000000000000000000000000 P e483b018db7b400ea03a6bcabfa946c9 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:19:18.118418 8644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:18.098259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32922 (local address 127.15.217.126:42961)
0810 07:19:18.098303 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:18.098306 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:18.105176 (+ 6870us) server_negotiation.cc:407] Connection header received
0810 07:19:18.105280 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:18.105281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:18.105293 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:18.105346 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:18.106194 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:18.106914 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:18.110325 (+ 3411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:18.110659 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:18.110706 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:18.112968 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:18.112970 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:18.112970 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:18.112991 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:18.114540 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:18.115871 (+ 1331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:18.115872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:18.115873 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:18.115936 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:18.117734 (+ 1798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:18.117736 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:18.117736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:18.117905 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:18.118017 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:18.118224 (+ 207us) server_negotiation.cc:299] Negotiation successful
0810 07:19:18.118313 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 07:19:18.120530 8723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:18.105157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34708 (local address 127.15.217.125:36253)
0810 07:19:18.105204 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:18.105206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:18.105358 (+ 152us) server_negotiation.cc:407] Connection header received
0810 07:19:18.105453 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:18.105455 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:18.105465 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:18.105508 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:18.108505 (+ 2997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:18.109248 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:18.111114 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:18.111454 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:18.111498 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:18.115563 (+ 4065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:18.115565 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:18.115565 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:18.115575 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:18.117252 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:18.118858 (+ 1606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:18.118860 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:18.118860 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:18.118930 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:18.119846 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:18.119847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:18.119847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:18.120029 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:18.120106 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:18.120316 (+ 210us) server_negotiation.cc:299] Negotiation successful
0810 07:19:18.120406 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 07:19:18.122812 8820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78994e8b371b423c90a9e64b38d3d591 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "78994e8b371b423c90a9e64b38d3d591" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78994e8b371b423c90a9e64b38d3d591" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42961 } } peers { permanent_uuid: "26a724e260724a48988c84c7ccfd6e98" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 36253 } } peers { permanent_uuid: "e483b018db7b400ea03a6bcabfa946c9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42721 } } }
I20260810 07:19:18.122938 8820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78994e8b371b423c90a9e64b38d3d591 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:18.122952 8719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42721 } attempt: 3
I20260810 07:19:18.124791 8806 sys_catalog.cc:455] T 00000000000000000000000000000000 P e483b018db7b400ea03a6bcabfa946c9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "78994e8b371b423c90a9e64b38d3d591" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78994e8b371b423c90a9e64b38d3d591" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42961 } } peers { permanent_uuid: "26a724e260724a48988c84c7ccfd6e98" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 36253 } } peers { permanent_uuid: "e483b018db7b400ea03a6bcabfa946c9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42721 } } }
I20260810 07:19:18.124897 8806 sys_catalog.cc:458] T 00000000000000000000000000000000 P e483b018db7b400ea03a6bcabfa946c9 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:19:18.128213 8804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:18.105042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53002 (local address 127.15.217.124:42721)
0810 07:19:18.105087 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:18.105089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:18.116636 (+ 11547us) server_negotiation.cc:407] Connection header received
0810 07:19:18.116706 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:18.116708 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:18.116723 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:18.116775 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:18.117530 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:18.118333 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:18.119403 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:18.119770 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:18.119834 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:18.121578 (+ 1744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:18.121579 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:18.121580 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:18.121605 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:18.123693 (+ 2088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:18.125172 (+ 1479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:18.125173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:18.125173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:18.125252 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:18.126464 (+ 1212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:18.126465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:18.126466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:18.126615 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:18.126707 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:18.128008 (+ 1301us) server_negotiation.cc:299] Negotiation successful
0810 07:19:18.128102 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":21}
Aug 10 07:19:18 dist-test-slave-10pc krb5kdc[8553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346357, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.217.125@KRBTEST.COM for kudu/127.15.217.124@KRBTEST.COM
I20260810 07:19:18.129539 8842 mvcc.cc:205] Tried to move back new op lower bound from 7316874682791485440 to 7316874682767753216. Current Snapshot: MvccSnapshot[applied={T|T < 7316874682791485440}]
I20260810 07:19:18.130999 8845 mvcc.cc:205] Tried to move back new op lower bound from 7316874682791485440 to 7316874682767753216. Current Snapshot: MvccSnapshot[applied={T|T < 7316874682791485440}]
I20260810 07:19:18.132347 8722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:18.123183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.217.124:42721 (local address 127.0.0.1:53014)
0810 07:19:18.123229 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:18.123240 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:18.123276 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:18.124026 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:18.124027 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:18.124206 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:18.124255 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:18.124256 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:18.125336 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:18.125337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:18.126162 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:18.126165 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:18.126741 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:18.126742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:18.126804 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:18.127404 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:18.127409 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:18.129037 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:18.131112 (+ 2075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:18.131113 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:18.131114 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:18.131353 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:18.131683 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:18.131684 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:18.131684 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:18.131742 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:18.132030 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:18.132031 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:18.132079 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 07:19:18.132141 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 07:19:18.132244 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260810 07:19:18.132856 8803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:18.123847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53014 (local address 127.15.217.124:42721)
0810 07:19:18.123890 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:18.123892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:18.123899 (+ 7us) server_negotiation.cc:407] Connection header received
0810 07:19:18.123918 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:18.123919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:18.123929 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:18.123977 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:18.124336 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:18.125129 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:18.126251 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:18.126563 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:18.126632 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:18.129120 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:18.129122 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:18.129122 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:18.129130 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:18.130900 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:18.131529 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:18.131530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:18.131531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:18.131608 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:18.131806 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:18.131807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:18.131808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:18.131933 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:18.132003 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:18.132696 (+ 693us) server_negotiation.cc:299] Negotiation successful
0810 07:19:18.132778 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 07:19:18.133284 8841 catalog_manager.cc:1347] Generated new cluster ID: 6154213ae7d940d693c91c2dcf2ce0d4
I20260810 07:19:18.133344 8841 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:18.134788 8719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26a724e260724a48988c84c7ccfd6e98: Bootstrap starting.
I20260810 07:19:18.135586 8719 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26a724e260724a48988c84c7ccfd6e98: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:18.135911 8719 log.cc:824] T 00000000000000000000000000000000 P 26a724e260724a48988c84c7ccfd6e98: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:18.136811 8719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26a724e260724a48988c84c7ccfd6e98: No bootstrap required, opened a new log
I20260810 07:19:18.139439 8719 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26a724e260724a48988c84c7ccfd6e98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78994e8b371b423c90a9e64b38d3d591" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42961 } } peers { permanent_uuid: "26a724e260724a48988c84c7ccfd6e98" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 36253 } } peers { permanent_uuid: "e483b018db7b400ea03a6bcabfa946c9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42721 } }
I20260810 07:19:18.139591 8719 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26a724e260724a48988c84c7ccfd6e98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26a724e260724a48988c84c7ccfd6e98, State: Initialized, Role: FOLLOWER
I20260810 07:19:18.139672 8719 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26a724e260724a48988c84c7ccfd6e98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78994e8b371b423c90a9e64b38d3d591" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42961 } } peers { permanent_uuid: "26a724e260724a48988c84c7ccfd6e98" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 36253 } } peers { permanent_uuid: "e483b018db7b400ea03a6bcabfa946c9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42721 } }
I20260810 07:19:18.140015 8719 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26a724e260724a48988c84c7ccfd6e98 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:18.140173 8690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "78994e8b371b423c90a9e64b38d3d591" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "26a724e260724a48988c84c7ccfd6e98"
I20260810 07:19:18.140643 8689 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26a724e260724a48988c84c7ccfd6e98 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:18.141645 8689 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 26a724e260724a48988c84c7ccfd6e98 [term 1 FOLLOWER]: Refusing update from remote peer 78994e8b371b423c90a9e64b38d3d591: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:18.142026 8820 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 78994e8b371b423c90a9e64b38d3d591 [LEADER]: Connected to new peer: Peer: permanent_uuid: "26a724e260724a48988c84c7ccfd6e98" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 36253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:18.142678 8690 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 26a724e260724a48988c84c7ccfd6e98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78994e8b371b423c90a9e64b38d3d591 in term 1.
I20260810 07:19:18.145604 8851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26a724e260724a48988c84c7ccfd6e98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78994e8b371b423c90a9e64b38d3d591" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42961 } } peers { permanent_uuid: "26a724e260724a48988c84c7ccfd6e98" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 36253 } } peers { permanent_uuid: "e483b018db7b400ea03a6bcabfa946c9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42721 } } }
I20260810 07:19:18.145787 8851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26a724e260724a48988c84c7ccfd6e98 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:19:18.145881 8851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26a724e260724a48988c84c7ccfd6e98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78994e8b371b423c90a9e64b38d3d591. Latest consensus state: current_term: 1 leader_uuid: "78994e8b371b423c90a9e64b38d3d591" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78994e8b371b423c90a9e64b38d3d591" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42961 } } peers { permanent_uuid: "26a724e260724a48988c84c7ccfd6e98" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 36253 } } peers { permanent_uuid: "e483b018db7b400ea03a6bcabfa946c9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42721 } } }
I20260810 07:19:18.145992 8851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26a724e260724a48988c84c7ccfd6e98 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:19:18.146062 8851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26a724e260724a48988c84c7ccfd6e98 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "78994e8b371b423c90a9e64b38d3d591" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78994e8b371b423c90a9e64b38d3d591" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42961 } } peers { permanent_uuid: "26a724e260724a48988c84c7ccfd6e98" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 36253 } } peers { permanent_uuid: "e483b018db7b400ea03a6bcabfa946c9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42721 } } }
I20260810 07:19:18.146150 8851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26a724e260724a48988c84c7ccfd6e98 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 07:19:18.148809 8863 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 26a724e260724a48988c84c7ccfd6e98: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:19:18.148890 8863 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:19:18.152977 8865 mvcc.cc:205] Tried to move back new op lower bound from 7316874682791485440 to 7316874682767753216. Current Snapshot: MvccSnapshot[applied={T|T < 7316874682791485440}]
I20260810 07:19:18.164573 8820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78994e8b371b423c90a9e64b38d3d591 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "78994e8b371b423c90a9e64b38d3d591" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78994e8b371b423c90a9e64b38d3d591" member_type: VOTER last_known_addr { host: "127.15.217.126" port: 42961 } } peers { permanent_uuid: "26a724e260724a48988c84c7ccfd6e98" member_type: VOTER last_known_addr { host: "127.15.217.125" port: 36253 } } peers { permanent_uuid: "e483b018db7b400ea03a6bcabfa946c9" member_type: VOTER last_known_addr { host: "127.15.217.124" port: 42721 } } }
I20260810 07:19:18.164719 8820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78994e8b371b423c90a9e64b38d3d591 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:18.167382 8841 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:18.168155 8841 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:18.179527 8841 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 78994e8b371b423c90a9e64b38d3d591: Generated new TSK 0
I20260810 07:19:18.179780 8841 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 07:19:18.208005 8723 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:18.206158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34724 (local address 127.15.217.125:36253)
0810 07:19:18.206210 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:18.206212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:18.207262 (+ 1050us) server_negotiation.cc:407] Connection header received
0810 07:19:18.207377 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:18.207378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:18.207390 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:18.207421 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:18.207923 (+ 502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34724: BlockingRecv error: recv got EOF from 127.0.0.1:34724 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260810 07:19:18.208912 8803 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:18.206579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53022 (local address 127.15.217.124:42721)
0810 07:19:18.206623 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:18.206625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:18.208238 (+ 1613us) server_negotiation.cc:407] Connection header received
0810 07:19:18.208360 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:18.208362 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:18.208374 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:18.208404 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:18.208848 (+ 444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53022: BlockingRecv error: recv got EOF from 127.0.0.1:53022 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260810 07:19:18.209697 8644 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:18.206963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32938 (local address 127.15.217.126:42961)
0810 07:19:18.207015 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:18.207017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:18.209154 (+ 2137us) server_negotiation.cc:407] Connection header received
0810 07:19:18.209252 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:18.209254 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:18.209265 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:18.209298 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:18.209631 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32938: BlockingRecv error: recv got EOF from 127.0.0.1:32938 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260810 07:19:18.213676 8723 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:18.212787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34736 (local address 127.15.217.125:36253)
0810 07:19:18.212831 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:18.212833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:18.213041 (+ 208us) server_negotiation.cc:407] Connection header received
0810 07:19:18.213164 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:18.213165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:18.213188 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:18.213218 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:18.213616 (+ 398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34736: BlockingRecv error: recv got EOF from 127.0.0.1:34736 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260810 07:19:18.217260 8723 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:18.216170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34748 (local address 127.15.217.125:36253)
0810 07:19:18.216211 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:18.216213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:18.216631 (+ 418us) server_negotiation.cc:407] Connection header received
0810 07:19:18.216742 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:18.216743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:18.216755 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:18.216785 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:18.217203 (+ 418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34748: BlockingRecv error: recv got EOF from 127.0.0.1:34748 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260810 07:19:18.221560 8723 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:18.220632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34750 (local address 127.15.217.125:36253)
0810 07:19:18.220676 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:18.220677 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:18.220962 (+ 285us) server_negotiation.cc:407] Connection header received
0810 07:19:18.221068 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:18.221069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:18.221081 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:18.221112 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:18.221488 (+ 376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34750: BlockingRecv error: recv got EOF from 127.0.0.1:34750 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260810 07:19:18.227739 8723 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:18.226791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34760 (local address 127.15.217.125:36253)
0810 07:19:18.226836 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:18.226838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:18.227131 (+ 293us) server_negotiation.cc:407] Connection header received
0810 07:19:18.227240 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:18.227242 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:18.227254 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:18.227283 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:18.227680 (+ 397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34760: BlockingRecv error: recv got EOF from 127.0.0.1:34760 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260810 07:19:18.238152 8723 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:18.237129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34774 (local address 127.15.217.125:36253)
0810 07:19:18.237178 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:18.237180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:18.237514 (+ 334us) server_negotiation.cc:407] Connection header received
0810 07:19:18.237650 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:18.237652 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:18.237664 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:18.237695 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:18.238087 (+ 392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34774: BlockingRecv error: recv got EOF from 127.0.0.1:34774 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260810 07:19:18.256896 8723 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:18.255602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34780 (local address 127.15.217.125:36253)
0810 07:19:18.255664 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:18.255666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:18.256001 (+ 335us) server_negotiation.cc:407] Connection header received
0810 07:19:18.256125 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:18.256127 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:18.256141 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:18.256173 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:18.256831 (+ 658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34780: BlockingRecv error: recv got EOF from 127.0.0.1:34780 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260810 07:19:18.291545 8723 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:18.290682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34790 (local address 127.15.217.125:36253)
0810 07:19:18.290736 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:18.290738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:18.290915 (+ 177us) server_negotiation.cc:407] Connection header received
0810 07:19:18.290992 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:18.290994 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:18.291008 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:18.291091 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:18.291481 (+ 390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34790: BlockingRecv error: recv got EOF from 127.0.0.1:34790 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260810 07:19:18.357602 8723 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:18.356799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34796 (local address 127.15.217.125:36253)
0810 07:19:18.356861 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:18.356868 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:18.356996 (+ 128us) server_negotiation.cc:407] Connection header received
0810 07:19:18.357040 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:18.357042 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:18.357053 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:18.357122 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:18.357532 (+ 410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34796: BlockingRecv error: recv got EOF from 127.0.0.1:34796 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260810 07:19:18.487905 8723 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:18.487078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34800 (local address 127.15.217.125:36253)
0810 07:19:18.487180 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:18.487183 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:18.487296 (+ 113us) server_negotiation.cc:407] Connection header received
0810 07:19:18.487348 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:18.487349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:18.487361 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:18.487417 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:18.487803 (+ 386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34800: BlockingRecv error: recv got EOF from 127.0.0.1:34800 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260810 07:19:18.745895 8723 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:18.745200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34810 (local address 127.15.217.125:36253)
0810 07:19:18.745286 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:18.745289 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:18.745302 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:19:18.745376 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:18.745377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:18.745389 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:18.745462 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:18.745835 (+ 373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34810: BlockingRecv error: recv got EOF from 127.0.0.1:34810 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 07:19:19.091557 8817 catalog_manager.cc:1259] Loaded cluster ID: 6154213ae7d940d693c91c2dcf2ce0d4
I20260810 07:19:19.091683 8817 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e483b018db7b400ea03a6bcabfa946c9: loading cluster ID for follower catalog manager: success
I20260810 07:19:19.092620 8817 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e483b018db7b400ea03a6bcabfa946c9: acquiring CA information for follower catalog manager: success
I20260810 07:19:19.092983 8817 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e483b018db7b400ea03a6bcabfa946c9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:19:19.149626 8863 catalog_manager.cc:1259] Loaded cluster ID: 6154213ae7d940d693c91c2dcf2ce0d4
I20260810 07:19:19.149724 8863 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 26a724e260724a48988c84c7ccfd6e98: loading cluster ID for follower catalog manager: success
I20260810 07:19:19.151150 8863 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 26a724e260724a48988c84c7ccfd6e98: acquiring CA information for follower catalog manager: success
I20260810 07:19:19.151571 8863 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 26a724e260724a48988c84c7ccfd6e98: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:19:19.262342 8940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:19.259250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34816 (local address 127.15.217.125:36253)
0810 07:19:19.259410 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:19.259412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:19.259448 (+ 36us) server_negotiation.cc:407] Connection header received
0810 07:19:19.259538 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:19.259539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:19.259598 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:19.259683 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:19.259892 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:19.260676 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:19.261436 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:19.261650 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:19.261706 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:19.261892 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:19.262046 (+ 154us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:19.262122 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:19.262174 (+ 52us) server_negotiation.cc:299] Negotiation successful
0810 07:19:19.262198 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":67,"threads_started":1}
I20260810 07:19:19.266845 8947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:19.264408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53030 (local address 127.15.217.124:42721)
0810 07:19:19.264571 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:19.264573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:19.264581 (+ 8us) server_negotiation.cc:407] Connection header received
0810 07:19:19.264622 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:19.264623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:19.264661 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:19.264732 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:19.264844 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:19.265383 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:19.266086 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:19.266285 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:19.266333 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:19.266496 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:19.266617 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:19.266673 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:19.266712 (+ 39us) server_negotiation.cc:299] Negotiation successful
0810 07:19:19.266729 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":108,"threads_started":1}
I20260810 07:19:19.271651 8947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:19.268830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53036 (local address 127.15.217.124:42721)
0810 07:19:19.268869 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:19.268871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:19.269009 (+ 138us) server_negotiation.cc:407] Connection header received
0810 07:19:19.269098 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:19.269099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:19.269110 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:19.269159 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:19.269315 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:19.270043 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:19.270788 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:19.271099 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:19.271164 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:19.271309 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:19.271341 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:19.271389 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:19.271535 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 07:19:19.271564 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 07:19:19.271765 8940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:19.268591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34820 (local address 127.15.217.125:36253)
0810 07:19:19.268641 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:19.268643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:19.268832 (+ 189us) server_negotiation.cc:407] Connection header received
0810 07:19:19.268929 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:19.268930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:19.268944 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:19.269006 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:19.269136 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:19.269894 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:19.270893 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:19.271195 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:19.271247 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:19.271422 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:19.271474 (+ 52us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:19.271506 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:19.271622 (+ 116us) server_negotiation.cc:299] Negotiation successful
0810 07:19:19.271650 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 07:19:19.271981 8954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:19.268653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32946 (local address 127.15.217.126:42961)
0810 07:19:19.268767 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:19.268769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:19.268896 (+ 127us) server_negotiation.cc:407] Connection header received
0810 07:19:19.268978 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:19.268979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:19.269032 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:19.269110 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:19.269305 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:19.269816 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:19.270913 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:19.271151 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:19.271211 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:19.271429 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:19.271545 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:19.271612 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:19.271870 (+ 258us) server_negotiation.cc:299] Negotiation successful
0810 07:19:19.271896 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":57,"threads_started":1}
I20260810 07:19:19.273809 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 8566
I20260810 07:19:19.286937 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 8647
I20260810 07:19:19.295603 16229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvoWU8l/build/release/bin/kudu with pid 8726
2026-08-10T07:19:19Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2924 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5880 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (277544 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST